TsTabletManagerITest.TestTableStats: /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/ts_tablet_manager-itest.cc:784: Failure
Expected equality of these values:
  live_row_count
    Which is: 802
  table_info->GetMetrics()->live_row_count->value()
    Which is: 1604
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/test_util.cc:403: Failure
Failed
Timed out waiting for assertion to pass.
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/ts_tablet_manager-itest.cc:768: Failure
Expected: check_function(table_infos[0].get(), live_row_count) doesn't generate new fatal failures in the current thread.
  Actual: it does.
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/ts_tablet_manager-itest.cc:786: Failure
Expected: GetLeaderMasterAndRun(live_row_count, [&] ( TableInfo* table_info, int64_t live_row_count) { do { AssertEventually([&] () { switch (0) case 0: default: if (const ::testing::AssertionResult gtest_ar_ = ::testing::AssertionResult(table_info->GetMetrics()->TableSupportsLiveRowCount())) ; else return ::testing::internal::AssertHelper(::testing::TestPartResult::kFatalFailure, "/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/ts_tablet_manager-itest.cc", 783, ::testing::internal::GetBoolAssertionFailureMessage( gtest_ar_, "table_info->GetMetrics()->TableSupportsLiveRowCount()", "false", "true") .c_str()) = ::testing::Message(); switch (0) case 0: default: if (const ::testing::AssertionResult gtest_ar = (::testing::internal::EqHelper::Compare("live_row_count", "table_info->GetMetrics()->live_row_count->value()", live_row_count, table_info->GetMetrics()->live_row_count->value()))) ; else return ::testing::internal::AssertHelper(::testing::TestPartResult::kFatalFailure, "/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/ts_tablet_manager-itest.cc", 784, gtest_ar.failure_message()) = ::testing::Message(); }); do { if (testing::Test::HasFatalFailure()) { return; } } while (0); } while (0); }) doesn't generate new fatal failures in the current thread.
  Actual: it does.
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/ts_tablet_manager-itest.cc:896: Failure
Expected: CheckStats(kRowsCount) doesn't generate new fatal failures in the current thread.
  Actual: it does.
I20251023 08:00:47.821589 18906 tablet_server.cc:178] TabletServer@127.18.118.129:0 shutting down...
W20251023 08:00:47.828727 20673 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 2 similar messages]
W20251023 08:00:47.840682 20673 consensus_peers.cc:597] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb -> Peer 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549): Couldn't send request to peer 4fd4035d6b864683b312d9eba4de5a3e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20251023 08:00:47.842104 20673 consensus_peers.cc:597] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb -> Peer 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549): Couldn't send request to peer 4fd4035d6b864683b312d9eba4de5a3e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20251023 08:00:47.858978 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:47.859999 18906 tablet_replica.cc:333] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e: stopping tablet replica
I20251023 08:00:47.860986 18906 raft_consensus.cc:2243] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:47.861987 18906 raft_consensus.cc:2272] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:47.865335 18906 tablet_replica.cc:333] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e: stopping tablet replica
I20251023 08:00:47.865859 18906 raft_consensus.cc:2243] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:47.866304 18906 raft_consensus.cc:2272] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:47.869248 18906 tablet_replica.cc:333] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e: stopping tablet replica
I20251023 08:00:47.869742 18906 raft_consensus.cc:2243] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 4 LEADER]: Raft consensus shutting down.
I20251023 08:00:47.870568 18906 raft_consensus.cc:2272] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:47.894007 18906 tablet_server.cc:195] TabletServer@127.18.118.129:0 shutdown complete.
I20251023 08:00:47.908049 18906 tablet_server.cc:178] TabletServer@127.18.118.130:0 shutting down...
I20251023 08:00:47.955819 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:47.956552 18906 tablet_replica.cc:333] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: stopping tablet replica
I20251023 08:00:47.957350 18906 raft_consensus.cc:2243] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 4 LEADER]: Raft consensus shutting down.
I20251023 08:00:47.958585 18906 raft_consensus.cc:2272] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:47.962136 18906 tablet_replica.cc:333] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: stopping tablet replica
I20251023 08:00:47.962615 18906 raft_consensus.cc:2243] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:47.963047 18906 raft_consensus.cc:2272] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:47.965883 18906 tablet_replica.cc:333] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: stopping tablet replica
I20251023 08:00:47.966339 18906 raft_consensus.cc:2243] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 4 LEADER]: Raft consensus shutting down.
I20251023 08:00:47.967186 18906 raft_consensus.cc:2272] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:47.993058 18906 tablet_server.cc:195] TabletServer@127.18.118.130:0 shutdown complete.
I20251023 08:00:48.007465 18906 tablet_server.cc:178] TabletServer@127.18.118.131:0 shutting down...
I20251023 08:00:48.040421 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:48.041247 18906 tablet_replica.cc:333] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e: stopping tablet replica
I20251023 08:00:48.041857 18906 raft_consensus.cc:2243] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:48.042526 18906 raft_consensus.cc:2272] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:48.045408 18906 tablet_replica.cc:333] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e: stopping tablet replica
I20251023 08:00:48.045867 18906 raft_consensus.cc:2243] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:48.046312 18906 raft_consensus.cc:2272] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:48.048748 18906 tablet_replica.cc:333] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e: stopping tablet replica
I20251023 08:00:48.049268 18906 raft_consensus.cc:2243] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:48.049800 18906 raft_consensus.cc:2272] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:48.096824 18906 tablet_server.cc:195] TabletServer@127.18.118.131:0 shutdown complete.
I20251023 08:00:48.110162 18906 master.cc:561] Master@127.18.118.190:38225 shutting down...
I20251023 08:00:48.139822 18906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 6 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:48.140493 18906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 6 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:48.140817 18906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a: stopping tablet replica
I20251023 08:00:48.183782 18906 master.cc:583] Master@127.18.118.190:38225 shutdown complete.
I20251023 08:00:48.199435 18906 master.cc:561] Master@127.18.118.189:44091 shutting down...
W20251023 08:00:48.214040 20967 consensus_peers.cc:597] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 -> Peer 8080b9557fda47c0af876ca344e2477a (127.18.118.190:38225): Couldn't send request to peer 8080b9557fda47c0af876ca344e2477a. Status: Network error: Client connection negotiation failed: client connection to 127.18.118.190:38225: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 08:00:48.228492 18906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 6 LEADER]: Raft consensus shutting down.
I20251023 08:00:48.229545 18906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:48.229889 18906 tablet_replica.cc:333] stopping tablet replica
I20251023 08:00:48.252323 18906 master.cc:583] Master@127.18.118.189:44091 shutdown complete.
I20251023 08:00:48.269349 18906 master.cc:561] Master@127.18.118.188:39171 shutting down...
I20251023 08:00:48.290781 18906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 6 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:48.291347 18906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:48.291630 18906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58: stopping tablet replica
I20251023 08:00:48.310581 18906 master.cc:583] Master@127.18.118.188:39171 shutdown complete.
I20251023 08:00:48.338570 18906 test_util.cc:183] -----------------------------------------------
I20251023 08:00:48.338833 18906 test_util.cc:184] Had failures, leaving test files at /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0

[==========] Running 13 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 3 tests from LeadershipChangeReportingTest
[ RUN      ] LeadershipChangeReportingTest.TestConcurrentGetTableLocations
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20251023 07:59:58.181136 18906 thread.cc:641] rpc reactor (reactor) Time spent creating pthread: real 1.162s	user 0.434s	sys 0.725s
W20251023 07:59:58.181860 18906 thread.cc:608] rpc reactor (reactor) Time spent starting thread: real 1.163s	user 0.434s	sys 0.728s
I20251023 07:59:58.184986 18906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.18.118.190:39183
I20251023 07:59:58.187454 18906 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251023 07:59:58.189332 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 07:59:58.205542 18917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 07:59:58.208047 18918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 07:59:58.285598 18920 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 07:59:58.287050 18906 server_base.cc:1047] running on GCE node
I20251023 07:59:58.288128 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 07:59:58.288357 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 07:59:58.288550 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206398288530 us; error 0 us; skew 500 ppm
I20251023 07:59:58.289292 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 07:59:58.294024 18906 webserver.cc:492] Webserver started at http://127.18.118.190:38049/ using document root <none> and password file <none>
I20251023 07:59:58.294838 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 07:59:58.295053 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 07:59:58.295462 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 07:59:58.301563 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1761206396810293-18906-0/minicluster-data/master-0-root/instance:
uuid: "296a29413a014f899f4b775e763bb8fe"
format_stamp: "Formatted at 2025-10-23 07:59:58 on dist-test-slave-wkn7"
I20251023 07:59:58.309309 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.006s	sys 0.001s
I20251023 07:59:58.314270 18925 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 07:59:58.315557 18906 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.005s	sys 0.000s
I20251023 07:59:58.315912 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1761206396810293-18906-0/minicluster-data/master-0-root
uuid: "296a29413a014f899f4b775e763bb8fe"
format_stamp: "Formatted at 2025-10-23 07:59:58 on dist-test-slave-wkn7"
I20251023 07:59:58.316233 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1761206396810293-18906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1761206396810293-18906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1761206396810293-18906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 07:59:58.352761 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 07:59:58.354420 18906 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251023 07:59:58.354868 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 07:59:58.425511 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.190:39183
I20251023 07:59:58.425722 18976 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.190:39183 every 8 connection(s)
I20251023 07:59:58.432019 18977 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 07:59:58.457743 18977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe: Bootstrap starting.
I20251023 07:59:58.465019 18977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe: Neither blocks nor log segments found. Creating new log.
I20251023 07:59:58.466823 18977 log.cc:826] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe: Log is configured to *not* fsync() on all Append() calls
I20251023 07:59:58.471155 18977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe: No bootstrap required, opened a new log
I20251023 07:59:58.492342 18977 raft_consensus.cc:359] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "296a29413a014f899f4b775e763bb8fe" member_type: VOTER }
I20251023 07:59:58.492908 18977 raft_consensus.cc:385] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 07:59:58.493198 18977 raft_consensus.cc:740] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 296a29413a014f899f4b775e763bb8fe, State: Initialized, Role: FOLLOWER
I20251023 07:59:58.493937 18977 consensus_queue.cc:260] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "296a29413a014f899f4b775e763bb8fe" member_type: VOTER }
I20251023 07:59:58.494426 18977 raft_consensus.cc:399] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 07:59:58.494694 18977 raft_consensus.cc:493] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 07:59:58.495021 18977 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe [term 0 FOLLOWER]: Advancing to term 1
I20251023 07:59:58.499094 18977 raft_consensus.cc:515] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "296a29413a014f899f4b775e763bb8fe" member_type: VOTER }
I20251023 07:59:58.499871 18977 leader_election.cc:304] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 296a29413a014f899f4b775e763bb8fe; no voters: 
I20251023 07:59:58.502027 18977 leader_election.cc:290] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe [CANDIDATE]: Term 1 election: Requested vote from peers 
I20251023 07:59:58.502328 18980 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe [term 1 FOLLOWER]: Leader election won for term 1
I20251023 07:59:58.505070 18980 raft_consensus.cc:697] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe [term 1 LEADER]: Becoming Leader. State: Replica: 296a29413a014f899f4b775e763bb8fe, State: Running, Role: LEADER
I20251023 07:59:58.505940 18980 consensus_queue.cc:237] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "296a29413a014f899f4b775e763bb8fe" member_type: VOTER }
I20251023 07:59:58.507248 18977 sys_catalog.cc:565] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe [sys.catalog]: configured and running, proceeding with master startup.
I20251023 07:59:58.513898 18981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "296a29413a014f899f4b775e763bb8fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "296a29413a014f899f4b775e763bb8fe" member_type: VOTER } }
I20251023 07:59:58.514626 18981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe [sys.catalog]: This master's current role is: LEADER
I20251023 07:59:58.516512 18982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 296a29413a014f899f4b775e763bb8fe. Latest consensus state: current_term: 1 leader_uuid: "296a29413a014f899f4b775e763bb8fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "296a29413a014f899f4b775e763bb8fe" member_type: VOTER } }
I20251023 07:59:58.516925 18982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe [sys.catalog]: This master's current role is: LEADER
I20251023 07:59:58.517978 18986 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 07:59:58.531250 18986 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 07:59:58.535539 18906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 07:59:58.548776 18986 catalog_manager.cc:1357] Generated new cluster ID: ed8ebb3318ec40b2aa5e2c86cb183cd7
I20251023 07:59:58.549129 18986 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 07:59:58.567155 18986 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 07:59:58.568737 18986 catalog_manager.cc:1514] Loading token signing keys...
I20251023 07:59:58.584182 18986 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe: Generated new TSK 0
I20251023 07:59:58.585186 18986 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 07:59:58.602223 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 07:59:58.615944 18999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 07:59:58.617884 19000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 07:59:58.623515 19002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 07:59:58.623651 18906 server_base.cc:1047] running on GCE node
I20251023 07:59:58.624531 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 07:59:58.624739 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 07:59:58.624897 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206398624880 us; error 0 us; skew 500 ppm
I20251023 07:59:58.625562 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 07:59:58.629945 18906 webserver.cc:492] Webserver started at http://127.18.118.129:41721/ using document root <none> and password file <none>
I20251023 07:59:58.630460 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 07:59:58.630658 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 07:59:58.630923 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 07:59:58.632140 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1761206396810293-18906-0/minicluster-data/ts-0-root/instance:
uuid: "85fb543c3650402092d9b83e1ee6bebb"
format_stamp: "Formatted at 2025-10-23 07:59:58 on dist-test-slave-wkn7"
I20251023 07:59:58.638365 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.000s	sys 0.008s
I20251023 07:59:58.642691 19007 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 07:59:58.643592 18906 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20251023 07:59:58.643884 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1761206396810293-18906-0/minicluster-data/ts-0-root
uuid: "85fb543c3650402092d9b83e1ee6bebb"
format_stamp: "Formatted at 2025-10-23 07:59:58 on dist-test-slave-wkn7"
I20251023 07:59:58.644181 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1761206396810293-18906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1761206396810293-18906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1761206396810293-18906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 07:59:58.663383 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 07:59:58.664680 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 07:59:58.667441 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 07:59:58.672137 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 07:59:58.672371 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 07:59:58.672638 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 07:59:58.672807 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 07:59:58.781985 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.129:37947
I20251023 07:59:58.782140 19069 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.129:37947 every 8 connection(s)
I20251023 07:59:58.788878 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 07:59:58.799377 19074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 07:59:58.803629 19075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 07:59:58.811239 18906 server_base.cc:1047] running on GCE node
W20251023 07:59:58.811956 19077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 07:59:58.813076 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 07:59:58.813356 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 07:59:58.813534 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206398813519 us; error 0 us; skew 500 ppm
I20251023 07:59:58.814353 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 07:59:58.816522 19070 heartbeater.cc:344] Connected to a master server at 127.18.118.190:39183
I20251023 07:59:58.816996 19070 heartbeater.cc:461] Registering TS with master...
I20251023 07:59:58.817605 18906 webserver.cc:492] Webserver started at http://127.18.118.130:43535/ using document root <none> and password file <none>
I20251023 07:59:58.818135 19070 heartbeater.cc:507] Master 127.18.118.190:39183 requested a full tablet report, sending...
I20251023 07:59:58.818254 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 07:59:58.818517 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 07:59:58.818951 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 07:59:58.820350 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1761206396810293-18906-0/minicluster-data/ts-1-root/instance:
uuid: "ca5af86f56c442c68b37179dc2f21108"
format_stamp: "Formatted at 2025-10-23 07:59:58 on dist-test-slave-wkn7"
I20251023 07:59:58.821198 18942 ts_manager.cc:194] Registered new tserver with Master: 85fb543c3650402092d9b83e1ee6bebb (127.18.118.129:37947)
I20251023 07:59:58.823313 18942 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53734
I20251023 07:59:58.827632 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.008s	sys 0.000s
I20251023 07:59:58.832703 19082 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 07:59:58.833626 18906 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20251023 07:59:58.833904 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1761206396810293-18906-0/minicluster-data/ts-1-root
uuid: "ca5af86f56c442c68b37179dc2f21108"
format_stamp: "Formatted at 2025-10-23 07:59:58 on dist-test-slave-wkn7"
I20251023 07:59:58.834182 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1761206396810293-18906-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1761206396810293-18906-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1761206396810293-18906-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 07:59:58.855463 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 07:59:58.856824 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 07:59:58.859551 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 07:59:58.863122 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 07:59:58.863315 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 07:59:58.863574 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 07:59:58.863725 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 07:59:58.927802 19070 heartbeater.cc:499] Master 127.18.118.190:39183 was elected leader, sending a full tablet report...
I20251023 07:59:58.960242 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.130:33125
I20251023 07:59:58.960786 19144 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.130:33125 every 8 connection(s)
I20251023 07:59:58.976929 19145 heartbeater.cc:344] Connected to a master server at 127.18.118.190:39183
I20251023 07:59:58.977421 19145 heartbeater.cc:461] Registering TS with master...
I20251023 07:59:58.978392 19145 heartbeater.cc:507] Master 127.18.118.190:39183 requested a full tablet report, sending...
I20251023 07:59:58.980602 18942 ts_manager.cc:194] Registered new tserver with Master: ca5af86f56c442c68b37179dc2f21108 (127.18.118.130:33125)
I20251023 07:59:58.980731 18906 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.015347291s
I20251023 07:59:58.982359 18942 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53748
I20251023 07:59:59.012298 18942 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53752:
name: "test-table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
  hash_schema {
    columns {
      name: "key"
    }
    num_buckets: 2
    seed: 0
  }
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20251023 07:59:59.041332 18942 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20251023 07:59:59.085255 19145 heartbeater.cc:499] Master 127.18.118.190:39183 was elected leader, sending a full tablet report...
I20251023 07:59:59.103917 19035 tablet_service.cc:1505] Processing CreateTablet for tablet 666f2ffbfa2148d6afaf0d34dd0072c2 (DEFAULT_TABLE table=test-table [id=6ab922d526ac411bb92988ad50a4fbc8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20251023 07:59:59.105608 19035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 666f2ffbfa2148d6afaf0d34dd0072c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 07:59:59.103969 19034 tablet_service.cc:1505] Processing CreateTablet for tablet 2c7d0eb9bffb46e6aae567a342820b1d (DEFAULT_TABLE table=test-table [id=6ab922d526ac411bb92988ad50a4fbc8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20251023 07:59:59.109711 19034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c7d0eb9bffb46e6aae567a342820b1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 07:59:59.111505 19110 tablet_service.cc:1505] Processing CreateTablet for tablet 666f2ffbfa2148d6afaf0d34dd0072c2 (DEFAULT_TABLE table=test-table [id=6ab922d526ac411bb92988ad50a4fbc8]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20251023 07:59:59.111953 19109 tablet_service.cc:1505] Processing CreateTablet for tablet 2c7d0eb9bffb46e6aae567a342820b1d (DEFAULT_TABLE table=test-table [id=6ab922d526ac411bb92988ad50a4fbc8]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20251023 07:59:59.112803 19110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 666f2ffbfa2148d6afaf0d34dd0072c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 07:59:59.114094 19109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c7d0eb9bffb46e6aae567a342820b1d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 07:59:59.139416 19158 tablet_bootstrap.cc:492] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108: Bootstrap starting.
I20251023 07:59:59.146807 19158 tablet_bootstrap.cc:654] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108: Neither blocks nor log segments found. Creating new log.
I20251023 07:59:59.148224 19159 tablet_bootstrap.cc:492] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb: Bootstrap starting.
I20251023 07:59:59.152235 19158 tablet_bootstrap.cc:492] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108: No bootstrap required, opened a new log
I20251023 07:59:59.152685 19158 ts_tablet_manager.cc:1403] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108: Time spent bootstrapping tablet: real 0.014s	user 0.011s	sys 0.000s
I20251023 07:59:59.154531 19159 tablet_bootstrap.cc:654] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb: Neither blocks nor log segments found. Creating new log.
I20251023 07:59:59.155784 19158 raft_consensus.cc:359] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } }
I20251023 07:59:59.156427 19158 raft_consensus.cc:740] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5af86f56c442c68b37179dc2f21108, State: Initialized, Role: FOLLOWER
I20251023 07:59:59.157059 19158 consensus_queue.cc:260] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } }
I20251023 07:59:59.160632 19158 ts_tablet_manager.cc:1434] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20251023 07:59:59.161247 19159 tablet_bootstrap.cc:492] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb: No bootstrap required, opened a new log
I20251023 07:59:59.161723 19159 ts_tablet_manager.cc:1403] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb: Time spent bootstrapping tablet: real 0.014s	user 0.009s	sys 0.003s
I20251023 07:59:59.161727 19158 tablet_bootstrap.cc:492] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108: Bootstrap starting.
I20251023 07:59:59.164819 19159 raft_consensus.cc:359] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } }
I20251023 07:59:59.165691 19159 raft_consensus.cc:740] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85fb543c3650402092d9b83e1ee6bebb, State: Initialized, Role: FOLLOWER
I20251023 07:59:59.166322 19159 consensus_queue.cc:260] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } }
I20251023 07:59:59.168783 19158 tablet_bootstrap.cc:654] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108: Neither blocks nor log segments found. Creating new log.
I20251023 07:59:59.186590 19158 tablet_bootstrap.cc:492] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108: No bootstrap required, opened a new log
I20251023 07:59:59.186738 19159 ts_tablet_manager.cc:1434] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb: Time spent starting tablet: real 0.025s	user 0.012s	sys 0.008s
I20251023 07:59:59.187088 19158 ts_tablet_manager.cc:1403] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.010s
I20251023 07:59:59.188066 19159 tablet_bootstrap.cc:492] T 666f2ffbfa2148d6afaf0d34dd0072c2 P 85fb543c3650402092d9b83e1ee6bebb: Bootstrap starting.
I20251023 07:59:59.189888 19158 raft_consensus.cc:359] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } }
I20251023 07:59:59.190549 19158 raft_consensus.cc:740] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca5af86f56c442c68b37179dc2f21108, State: Initialized, Role: FOLLOWER
I20251023 07:59:59.191133 19158 consensus_queue.cc:260] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } }
I20251023 07:59:59.193274 19158 ts_tablet_manager.cc:1434] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20251023 07:59:59.196535 19159 tablet_bootstrap.cc:654] T 666f2ffbfa2148d6afaf0d34dd0072c2 P 85fb543c3650402092d9b83e1ee6bebb: Neither blocks nor log segments found. Creating new log.
I20251023 07:59:59.202044 19159 tablet_bootstrap.cc:492] T 666f2ffbfa2148d6afaf0d34dd0072c2 P 85fb543c3650402092d9b83e1ee6bebb: No bootstrap required, opened a new log
I20251023 07:59:59.202426 19159 ts_tablet_manager.cc:1403] T 666f2ffbfa2148d6afaf0d34dd0072c2 P 85fb543c3650402092d9b83e1ee6bebb: Time spent bootstrapping tablet: real 0.015s	user 0.005s	sys 0.004s
I20251023 07:59:59.204468 19159 raft_consensus.cc:359] T 666f2ffbfa2148d6afaf0d34dd0072c2 P 85fb543c3650402092d9b83e1ee6bebb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } }
I20251023 07:59:59.204948 19159 raft_consensus.cc:740] T 666f2ffbfa2148d6afaf0d34dd0072c2 P 85fb543c3650402092d9b83e1ee6bebb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85fb543c3650402092d9b83e1ee6bebb, State: Initialized, Role: FOLLOWER
I20251023 07:59:59.205511 19159 consensus_queue.cc:260] T 666f2ffbfa2148d6afaf0d34dd0072c2 P 85fb543c3650402092d9b83e1ee6bebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } }
I20251023 07:59:59.207520 19159 ts_tablet_manager.cc:1434] T 666f2ffbfa2148d6afaf0d34dd0072c2 P 85fb543c3650402092d9b83e1ee6bebb: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
W20251023 07:59:59.216228 19146 tablet.cc:2378] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251023 07:59:59.241293 18906 ts_tablet_manager-itest.cc:368] Electing peer 2...
I20251023 07:59:59.241677 18906 raft_consensus.cc:423] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108 [term 0 FOLLOWER]: Emulating election...
I20251023 07:59:59.242002 18906 raft_consensus.cc:3060] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108 [term 0 FOLLOWER]: Advancing to term 1
I20251023 07:59:59.248325 18906 raft_consensus.cc:697] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108 [term 1 LEADER]: Becoming Leader. State: Replica: ca5af86f56c442c68b37179dc2f21108, State: Running, Role: LEADER
I20251023 07:59:59.249053 18906 consensus_queue.cc:237] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } }
I20251023 07:59:59.256049 18906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20251023 07:59:59.258836 18942 catalog_manager.cc:5649] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca5af86f56c442c68b37179dc2f21108 (127.18.118.130). New cstate: current_term: 1 leader_uuid: "ca5af86f56c442c68b37179dc2f21108" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } health_report { overall_health: HEALTHY } } }
I20251023 07:59:59.279194 18906 cluster_itest_util.cc:257] Not converged past 0 yet: 0.0 1.1
I20251023 07:59:59.383153 18906 cluster_itest_util.cc:257] Not converged past 0 yet: 0.0 1.1
I20251023 07:59:59.587323 18906 cluster_itest_util.cc:257] Not converged past 0 yet: 0.0 1.1
I20251023 07:59:59.831727 19045 raft_consensus.cc:3060] T 666f2ffbfa2148d6afaf0d34dd0072c2 P 85fb543c3650402092d9b83e1ee6bebb [term 0 FOLLOWER]: Advancing to term 1
I20251023 07:59:59.841064 19169 consensus_queue.cc:1048] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 07:59:59.891132 18906 ts_tablet_manager-itest.cc:368] Electing peer 3...
I20251023 07:59:59.891489 18906 raft_consensus.cc:423] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [term 0 FOLLOWER]: Emulating election...
I20251023 07:59:59.891726 18906 raft_consensus.cc:3060] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [term 0 FOLLOWER]: Advancing to term 1
I20251023 07:59:59.895360 18906 raft_consensus.cc:697] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [term 1 LEADER]: Becoming Leader. State: Replica: ca5af86f56c442c68b37179dc2f21108, State: Running, Role: LEADER
I20251023 07:59:59.896003 18906 consensus_queue.cc:237] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } }
I20251023 07:59:59.898766 18906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20251023 07:59:59.902668 18906 cluster_itest_util.cc:257] Not converged past 0 yet: 0.0 1.1
I20251023 07:59:59.902386 18940 catalog_manager.cc:5649] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 reported cstate change: term changed from 0 to 1, leader changed from <none> to ca5af86f56c442c68b37179dc2f21108 (127.18.118.130). New cstate: current_term: 1 leader_uuid: "ca5af86f56c442c68b37179dc2f21108" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } health_report { overall_health: HEALTHY } } }
I20251023 08:00:00.006356 18906 cluster_itest_util.cc:257] Not converged past 0 yet: 0.0 1.1
I20251023 08:00:00.210520 18906 cluster_itest_util.cc:257] Not converged past 0 yet: 0.0 1.1
I20251023 08:00:00.365900 19045 raft_consensus.cc:3060] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:00.371292 19174 consensus_queue.cc:1048] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 08:00:00.715307 18906 ts_tablet_manager-itest.cc:368] Electing peer 3...
I20251023 08:00:00.715549 18906 raft_consensus.cc:423] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [term 1 LEADER]: Emulating election...
I20251023 08:00:00.715793 18906 raft_consensus.cc:3055] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 08:00:00.715991 18906 raft_consensus.cc:740] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ca5af86f56c442c68b37179dc2f21108, State: Running, Role: LEADER
I20251023 08:00:00.716454 18906 consensus_queue.cc:260] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } }
I20251023 08:00:00.717212 18906 raft_consensus.cc:3060] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:00.720964 18906 raft_consensus.cc:697] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [term 2 LEADER]: Becoming Leader. State: Replica: ca5af86f56c442c68b37179dc2f21108, State: Running, Role: LEADER
I20251023 08:00:00.721586 18906 consensus_queue.cc:237] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } }
I20251023 08:00:00.723001 18906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20251023 08:00:00.726683 18906 cluster_itest_util.cc:257] Not converged past 0 yet: 1.1 2.2
I20251023 08:00:00.726372 18942 catalog_manager.cc:5649] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "ca5af86f56c442c68b37179dc2f21108" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } health_report { overall_health: HEALTHY } } }
I20251023 08:00:00.830533 18906 cluster_itest_util.cc:257] Not converged past 0 yet: 1.1 2.2
I20251023 08:00:01.034411 18906 cluster_itest_util.cc:257] Not converged past 0 yet: 1.1 2.2
I20251023 08:00:01.111896 19045 raft_consensus.cc:3060] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:01.116532 19045 raft_consensus.cc:1275] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [term 2 FOLLOWER]: Refusing update from remote peer ca5af86f56c442c68b37179dc2f21108: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 08:00:01.117839 19183 consensus_queue.cc:1048] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 08:00:01.338621 18906 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20251023 08:00:01.338943 18906 raft_consensus.cc:423] T 666f2ffbfa2148d6afaf0d34dd0072c2 P 85fb543c3650402092d9b83e1ee6bebb [term 1 FOLLOWER]: Emulating election...
I20251023 08:00:01.339221 18906 raft_consensus.cc:3060] T 666f2ffbfa2148d6afaf0d34dd0072c2 P 85fb543c3650402092d9b83e1ee6bebb [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:01.343087 18906 raft_consensus.cc:697] T 666f2ffbfa2148d6afaf0d34dd0072c2 P 85fb543c3650402092d9b83e1ee6bebb [term 2 LEADER]: Becoming Leader. State: Replica: 85fb543c3650402092d9b83e1ee6bebb, State: Running, Role: LEADER
I20251023 08:00:01.343771 18906 consensus_queue.cc:237] T 666f2ffbfa2148d6afaf0d34dd0072c2 P 85fb543c3650402092d9b83e1ee6bebb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } }
I20251023 08:00:01.345661 18906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20251023 08:00:01.349578 18941 catalog_manager.cc:5649] T 666f2ffbfa2148d6afaf0d34dd0072c2 P 85fb543c3650402092d9b83e1ee6bebb reported cstate change: term changed from 1 to 2, leader changed from ca5af86f56c442c68b37179dc2f21108 (127.18.118.130) to 85fb543c3650402092d9b83e1ee6bebb (127.18.118.129). New cstate: current_term: 2 leader_uuid: "85fb543c3650402092d9b83e1ee6bebb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } health_report { overall_health: UNKNOWN } } }
I20251023 08:00:01.350602 18906 cluster_itest_util.cc:257] Not converged past 0 yet: 2.2 1.1
I20251023 08:00:01.416579 19045 raft_consensus.cc:1240] T 666f2ffbfa2148d6afaf0d34dd0072c2 P 85fb543c3650402092d9b83e1ee6bebb [term 2 LEADER]: Rejecting Update request from peer ca5af86f56c442c68b37179dc2f21108 for earlier term 1. Current term is 2. Ops: []
I20251023 08:00:01.417708 19190 consensus_queue.cc:1059] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 08:00:01.418263 19190 consensus_queue.cc:1243] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108 [LEADER]: Peer 85fb543c3650402092d9b83e1ee6bebb log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20251023 08:00:01.418630 19183 raft_consensus.cc:3055] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 08:00:01.418875 19183 raft_consensus.cc:740] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ca5af86f56c442c68b37179dc2f21108, State: Running, Role: LEADER
I20251023 08:00:01.419394 19183 consensus_queue.cc:260] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } }
I20251023 08:00:01.420038 19183 raft_consensus.cc:3060] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108 [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:01.454350 18906 cluster_itest_util.cc:257] Not converged past 0 yet: 2.2 1.1
I20251023 08:00:01.658308 18906 cluster_itest_util.cc:257] Not converged past 0 yet: 2.2 1.1
I20251023 08:00:01.923974 19120 raft_consensus.cc:1275] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108 [term 2 FOLLOWER]: Refusing update from remote peer 85fb543c3650402092d9b83e1ee6bebb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 08:00:01.925267 19197 consensus_queue.cc:1048] T 666f2ffbfa2148d6afaf0d34dd0072c2 P 85fb543c3650402092d9b83e1ee6bebb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 08:00:02.162470 18906 ts_tablet_manager-itest.cc:368] Electing peer 1...
I20251023 08:00:02.162735 18906 raft_consensus.cc:423] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [term 2 FOLLOWER]: Emulating election...
I20251023 08:00:02.162947 18906 raft_consensus.cc:3060] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [term 2 FOLLOWER]: Advancing to term 3
I20251023 08:00:02.167060 18906 raft_consensus.cc:697] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [term 3 LEADER]: Becoming Leader. State: Replica: 85fb543c3650402092d9b83e1ee6bebb, State: Running, Role: LEADER
I20251023 08:00:02.167590 18906 consensus_queue.cc:237] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } }
I20251023 08:00:02.169270 18906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20251023 08:00:02.172672 18906 cluster_itest_util.cc:257] Not converged past 0 yet: 3.3 2.2
I20251023 08:00:02.172678 18941 catalog_manager.cc:5649] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb reported cstate change: term changed from 2 to 3, leader changed from ca5af86f56c442c68b37179dc2f21108 (127.18.118.130) to 85fb543c3650402092d9b83e1ee6bebb (127.18.118.129). New cstate: current_term: 3 leader_uuid: "85fb543c3650402092d9b83e1ee6bebb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } health_report { overall_health: UNKNOWN } } }
I20251023 08:00:02.197826 19045 raft_consensus.cc:1240] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [term 3 LEADER]: Rejecting Update request from peer ca5af86f56c442c68b37179dc2f21108 for earlier term 2. Current term is 3. Ops: []
I20251023 08:00:02.199090 19183 consensus_queue.cc:1059] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 08:00:02.200246 19183 consensus_queue.cc:1243] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [LEADER]: Peer 85fb543c3650402092d9b83e1ee6bebb log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20251023 08:00:02.200675 19208 raft_consensus.cc:3055] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [term 2 LEADER]: Stepping down as leader of term 2
I20251023 08:00:02.200947 19208 raft_consensus.cc:740] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ca5af86f56c442c68b37179dc2f21108, State: Running, Role: LEADER
I20251023 08:00:02.201480 19208 consensus_queue.cc:260] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } }
I20251023 08:00:02.202167 19208 raft_consensus.cc:3060] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [term 2 FOLLOWER]: Advancing to term 3
I20251023 08:00:02.276508 18906 cluster_itest_util.cc:257] Not converged past 0 yet: 3.3 2.2
I20251023 08:00:02.480088 18906 cluster_itest_util.cc:257] Not converged past 0 yet: 3.3 2.2
I20251023 08:00:02.744105 19120 raft_consensus.cc:1275] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [term 3 FOLLOWER]: Refusing update from remote peer 85fb543c3650402092d9b83e1ee6bebb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20251023 08:00:02.745199 19201 consensus_queue.cc:1048] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 08:00:02.783917 18906 ts_tablet_manager-itest.cc:368] Electing peer 3...
I20251023 08:00:02.784202 18906 raft_consensus.cc:423] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [term 3 FOLLOWER]: Emulating election...
I20251023 08:00:02.784430 18906 raft_consensus.cc:3060] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [term 3 FOLLOWER]: Advancing to term 4
I20251023 08:00:02.788336 18906 raft_consensus.cc:697] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [term 4 LEADER]: Becoming Leader. State: Replica: ca5af86f56c442c68b37179dc2f21108, State: Running, Role: LEADER
I20251023 08:00:02.788954 18906 consensus_queue.cc:237] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } }
I20251023 08:00:02.790520 18906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20251023 08:00:02.793928 18906 cluster_itest_util.cc:257] Not converged past 0 yet: 3.3 4.4
I20251023 08:00:02.793959 18941 catalog_manager.cc:5649] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 reported cstate change: term changed from 3 to 4, leader changed from 85fb543c3650402092d9b83e1ee6bebb (127.18.118.129) to ca5af86f56c442c68b37179dc2f21108 (127.18.118.130). New cstate: current_term: 4 leader_uuid: "ca5af86f56c442c68b37179dc2f21108" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } health_report { overall_health: HEALTHY } } }
I20251023 08:00:02.898211 18906 cluster_itest_util.cc:257] Not converged past 0 yet: 3.3 4.4
I20251023 08:00:03.101989 18906 cluster_itest_util.cc:257] Not converged past 0 yet: 3.3 4.4
I20251023 08:00:03.296759 19120 raft_consensus.cc:1240] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [term 4 LEADER]: Rejecting Update request from peer 85fb543c3650402092d9b83e1ee6bebb for earlier term 3. Current term is 4. Ops: []
I20251023 08:00:03.297930 19211 consensus_queue.cc:1059] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 08:00:03.298856 19211 consensus_queue.cc:1243] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [LEADER]: Peer ca5af86f56c442c68b37179dc2f21108 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20251023 08:00:03.299199 19217 raft_consensus.cc:3055] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [term 3 LEADER]: Stepping down as leader of term 3
I20251023 08:00:03.299527 19217 raft_consensus.cc:740] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 85fb543c3650402092d9b83e1ee6bebb, State: Running, Role: LEADER
I20251023 08:00:03.300096 19217 consensus_queue.cc:260] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 } } peers { permanent_uuid: "ca5af86f56c442c68b37179dc2f21108" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 33125 } }
I20251023 08:00:03.300854 19217 raft_consensus.cc:3060] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [term 3 FOLLOWER]: Advancing to term 4
I20251023 08:00:03.406049 18906 cluster_itest_util.cc:257] Not converged past 0 yet: 3.3 4.4
I20251023 08:00:03.407790 19045 raft_consensus.cc:1275] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [term 4 FOLLOWER]: Refusing update from remote peer ca5af86f56c442c68b37179dc2f21108: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20251023 08:00:03.409129 19218 consensus_queue.cc:1048] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85fb543c3650402092d9b83e1ee6bebb" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 37947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 08:00:04.013231 18906 tablet_server.cc:178] TabletServer@127.18.118.129:0 shutting down...
I20251023 08:00:04.032254 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:04.032900 18906 tablet_replica.cc:333] T 666f2ffbfa2148d6afaf0d34dd0072c2 P 85fb543c3650402092d9b83e1ee6bebb: stopping tablet replica
I20251023 08:00:04.033488 18906 raft_consensus.cc:2243] T 666f2ffbfa2148d6afaf0d34dd0072c2 P 85fb543c3650402092d9b83e1ee6bebb [term 2 LEADER]: Raft consensus shutting down.
I20251023 08:00:04.034083 18906 raft_consensus.cc:2272] T 666f2ffbfa2148d6afaf0d34dd0072c2 P 85fb543c3650402092d9b83e1ee6bebb [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:04.035928 18906 tablet_replica.cc:333] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb: stopping tablet replica
I20251023 08:00:04.036309 18906 raft_consensus.cc:2243] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:04.036621 18906 raft_consensus.cc:2272] T 2c7d0eb9bffb46e6aae567a342820b1d P 85fb543c3650402092d9b83e1ee6bebb [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:04.121667 18906 tablet_server.cc:195] TabletServer@127.18.118.129:0 shutdown complete.
I20251023 08:00:04.131134 18906 tablet_server.cc:178] TabletServer@127.18.118.130:0 shutting down...
I20251023 08:00:04.147673 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:04.148392 18906 tablet_replica.cc:333] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108: stopping tablet replica
I20251023 08:00:04.149045 18906 raft_consensus.cc:2243] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:04.149659 18906 raft_consensus.cc:2272] T 666f2ffbfa2148d6afaf0d34dd0072c2 P ca5af86f56c442c68b37179dc2f21108 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:04.151118 18906 tablet_replica.cc:333] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108: stopping tablet replica
I20251023 08:00:04.151563 18906 raft_consensus.cc:2243] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [term 4 LEADER]: Raft consensus shutting down.
I20251023 08:00:04.152115 18906 raft_consensus.cc:2272] T 2c7d0eb9bffb46e6aae567a342820b1d P ca5af86f56c442c68b37179dc2f21108 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:04.170524 18906 tablet_server.cc:195] TabletServer@127.18.118.130:0 shutdown complete.
I20251023 08:00:04.178347 18906 master.cc:561] Master@127.18.118.190:39183 shutting down...
I20251023 08:00:04.192449 18906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe [term 1 LEADER]: Raft consensus shutting down.
I20251023 08:00:04.193073 18906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:04.194036 18906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 296a29413a014f899f4b775e763bb8fe: stopping tablet replica
I20251023 08:00:04.211068 18906 master.cc:583] Master@127.18.118.190:39183 shutdown complete.
[       OK ] LeadershipChangeReportingTest.TestConcurrentGetTableLocations (7261 ms)
[ RUN      ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange
I20251023 08:00:04.237507 18906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.18.118.190:39265
I20251023 08:00:04.238425 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:04.242689 19227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:04.243788 19228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:04.246354 19230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:04.246704 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:04.248375 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:04.248778 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:04.248898 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206404248886 us; error 0 us; skew 500 ppm
I20251023 08:00:04.249531 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:04.251835 18906 webserver.cc:492] Webserver started at http://127.18.118.190:42895/ using document root <none> and password file <none>
I20251023 08:00:04.252238 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:04.252393 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:04.252599 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:04.253629 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1761206396810293-18906-0/minicluster-data/master-0-root/instance:
uuid: "5a30c6a030f24a72bb72ec68eedef081"
format_stamp: "Formatted at 2025-10-23 08:00:04 on dist-test-slave-wkn7"
I20251023 08:00:04.257661 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20251023 08:00:04.260859 19235 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:04.261619 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20251023 08:00:04.261890 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1761206396810293-18906-0/minicluster-data/master-0-root
uuid: "5a30c6a030f24a72bb72ec68eedef081"
format_stamp: "Formatted at 2025-10-23 08:00:04 on dist-test-slave-wkn7"
I20251023 08:00:04.262167 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1761206396810293-18906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1761206396810293-18906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1761206396810293-18906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:04.280229 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:04.281248 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:04.315757 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.190:39265
I20251023 08:00:04.315853 19286 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.190:39265 every 8 connection(s)
I20251023 08:00:04.319413 19287 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:04.328234 19287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081: Bootstrap starting.
I20251023 08:00:04.332016 19287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:04.335594 19287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081: No bootstrap required, opened a new log
I20251023 08:00:04.337373 19287 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a30c6a030f24a72bb72ec68eedef081" member_type: VOTER }
I20251023 08:00:04.337726 19287 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:04.337906 19287 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a30c6a030f24a72bb72ec68eedef081, State: Initialized, Role: FOLLOWER
I20251023 08:00:04.338380 19287 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a30c6a030f24a72bb72ec68eedef081" member_type: VOTER }
I20251023 08:00:04.338774 19287 raft_consensus.cc:399] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 08:00:04.338965 19287 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 08:00:04.339165 19287 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:04.342921 19287 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a30c6a030f24a72bb72ec68eedef081" member_type: VOTER }
I20251023 08:00:04.343374 19287 leader_election.cc:304] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a30c6a030f24a72bb72ec68eedef081; no voters: 
I20251023 08:00:04.344452 19287 leader_election.cc:290] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20251023 08:00:04.344730 19290 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:04.346033 19290 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081 [term 1 LEADER]: Becoming Leader. State: Replica: 5a30c6a030f24a72bb72ec68eedef081, State: Running, Role: LEADER
I20251023 08:00:04.346557 19290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a30c6a030f24a72bb72ec68eedef081" member_type: VOTER }
I20251023 08:00:04.347234 19287 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 08:00:04.349334 19291 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a30c6a030f24a72bb72ec68eedef081" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a30c6a030f24a72bb72ec68eedef081" member_type: VOTER } }
I20251023 08:00:04.349373 19292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a30c6a030f24a72bb72ec68eedef081. Latest consensus state: current_term: 1 leader_uuid: "5a30c6a030f24a72bb72ec68eedef081" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a30c6a030f24a72bb72ec68eedef081" member_type: VOTER } }
I20251023 08:00:04.350219 19292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:04.350085 19291 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:04.359818 19296 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 08:00:04.364468 19296 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 08:00:04.365597 18906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 08:00:04.372320 19296 catalog_manager.cc:1357] Generated new cluster ID: 970ad609480842b894f9733c86e5dbbb
I20251023 08:00:04.372588 19296 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 08:00:04.394130 19296 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 08:00:04.395316 19296 catalog_manager.cc:1514] Loading token signing keys...
I20251023 08:00:04.405663 19296 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081: Generated new TSK 0
I20251023 08:00:04.406246 19296 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 08:00:04.431710 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:04.437539 19308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:04.439121 19309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:04.440100 19311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:04.440837 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:04.441599 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:04.441772 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:04.441901 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206404441889 us; error 0 us; skew 500 ppm
I20251023 08:00:04.442322 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:04.444626 18906 webserver.cc:492] Webserver started at http://127.18.118.129:38243/ using document root <none> and password file <none>
I20251023 08:00:04.445056 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:04.445276 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:04.445493 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:04.446451 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1761206396810293-18906-0/minicluster-data/ts-0-root/instance:
uuid: "53e708574f514aeca9afe27b5c3fa257"
format_stamp: "Formatted at 2025-10-23 08:00:04 on dist-test-slave-wkn7"
I20251023 08:00:04.455368 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.010s	sys 0.000s
I20251023 08:00:04.459667 19316 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:04.460361 18906 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20251023 08:00:04.460631 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1761206396810293-18906-0/minicluster-data/ts-0-root
uuid: "53e708574f514aeca9afe27b5c3fa257"
format_stamp: "Formatted at 2025-10-23 08:00:04 on dist-test-slave-wkn7"
I20251023 08:00:04.460922 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1761206396810293-18906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1761206396810293-18906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1761206396810293-18906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:04.488526 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:04.489590 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:04.491089 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:00:04.493358 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:04.493551 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:04.493795 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:04.493947 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:04.531774 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.129:42555
I20251023 08:00:04.531895 19378 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.129:42555 every 8 connection(s)
I20251023 08:00:04.536252 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:04.543218 19383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:04.543483 19384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:04.546088 18906 server_base.cc:1047] running on GCE node
W20251023 08:00:04.546595 19386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:04.547371 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:04.547552 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:04.547715 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206404547698 us; error 0 us; skew 500 ppm
I20251023 08:00:04.548390 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:04.551169 18906 webserver.cc:492] Webserver started at http://127.18.118.130:35775/ using document root <none> and password file <none>
I20251023 08:00:04.551192 19379 heartbeater.cc:344] Connected to a master server at 127.18.118.190:39265
I20251023 08:00:04.551574 19379 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:04.551697 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:04.551930 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:04.552175 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:04.552261 19379 heartbeater.cc:507] Master 127.18.118.190:39265 requested a full tablet report, sending...
I20251023 08:00:04.553406 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1761206396810293-18906-0/minicluster-data/ts-1-root/instance:
uuid: "0417fce582ed4078bc474ff37a9facf0"
format_stamp: "Formatted at 2025-10-23 08:00:04 on dist-test-slave-wkn7"
I20251023 08:00:04.554515 19252 ts_manager.cc:194] Registered new tserver with Master: 53e708574f514aeca9afe27b5c3fa257 (127.18.118.129:42555)
I20251023 08:00:04.556093 19252 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34238
I20251023 08:00:04.557933 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.002s
I20251023 08:00:04.561031 19391 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:04.561839 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20251023 08:00:04.562126 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1761206396810293-18906-0/minicluster-data/ts-1-root
uuid: "0417fce582ed4078bc474ff37a9facf0"
format_stamp: "Formatted at 2025-10-23 08:00:04 on dist-test-slave-wkn7"
I20251023 08:00:04.562402 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1761206396810293-18906-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1761206396810293-18906-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1761206396810293-18906-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:04.592162 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:04.593276 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:04.594642 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:00:04.596727 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:04.596921 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:04.597179 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:04.597337 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:04.633514 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.130:42477
I20251023 08:00:04.633597 19453 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.130:42477 every 8 connection(s)
I20251023 08:00:04.645686 19454 heartbeater.cc:344] Connected to a master server at 127.18.118.190:39265
I20251023 08:00:04.646045 19454 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:04.646693 19454 heartbeater.cc:507] Master 127.18.118.190:39265 requested a full tablet report, sending...
I20251023 08:00:04.648451 19252 ts_manager.cc:194] Registered new tserver with Master: 0417fce582ed4078bc474ff37a9facf0 (127.18.118.130:42477)
I20251023 08:00:04.648603 18906 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.012160248s
I20251023 08:00:04.650141 19252 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34250
I20251023 08:00:04.658301 19379 heartbeater.cc:499] Master 127.18.118.190:39265 was elected leader, sending a full tablet report...
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/ts_tablet_manager-itest.cc:422: Skipped
test is skipped; set KUDU_ALLOW_SLOW_TESTS=1 to run
I20251023 08:00:04.671833 18906 tablet_server.cc:178] TabletServer@127.18.118.129:0 shutting down...
I20251023 08:00:04.688370 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:04.703522 18906 tablet_server.cc:195] TabletServer@127.18.118.129:0 shutdown complete.
I20251023 08:00:04.709821 18906 tablet_server.cc:178] TabletServer@127.18.118.130:0 shutting down...
I20251023 08:00:04.724012 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:04.738710 18906 tablet_server.cc:195] TabletServer@127.18.118.130:0 shutdown complete.
I20251023 08:00:04.744900 18906 master.cc:561] Master@127.18.118.190:39265 shutting down...
I20251023 08:00:04.822912 18906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081 [term 1 LEADER]: Raft consensus shutting down.
I20251023 08:00:04.823421 18906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:04.823818 18906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 5a30c6a030f24a72bb72ec68eedef081: stopping tablet replica
I20251023 08:00:04.832507 18906 master.cc:583] Master@127.18.118.190:39265 shutdown complete.
[  SKIPPED ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange (615 ms)
[ RUN      ] LeadershipChangeReportingTest.TestUpdatedConsensusState
I20251023 08:00:04.853037 18906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.18.118.190:39749
I20251023 08:00:04.854091 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:04.858596 19467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:04.859639 19468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:04.862337 19470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:04.862378 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:04.863246 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:04.863433 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:04.863580 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206404863563 us; error 0 us; skew 500 ppm
I20251023 08:00:04.864041 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:04.866402 18906 webserver.cc:492] Webserver started at http://127.18.118.190:40251/ using document root <none> and password file <none>
I20251023 08:00:04.866852 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:04.867034 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:04.867286 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:04.868283 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1761206396810293-18906-0/minicluster-data/master-0-root/instance:
uuid: "7c6385fc0bf24edf9b1b27c711857371"
format_stamp: "Formatted at 2025-10-23 08:00:04 on dist-test-slave-wkn7"
I20251023 08:00:04.872320 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.005s
I20251023 08:00:04.875289 19475 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:04.875998 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20251023 08:00:04.876222 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1761206396810293-18906-0/minicluster-data/master-0-root
uuid: "7c6385fc0bf24edf9b1b27c711857371"
format_stamp: "Formatted at 2025-10-23 08:00:04 on dist-test-slave-wkn7"
I20251023 08:00:04.876452 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1761206396810293-18906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1761206396810293-18906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1761206396810293-18906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:04.892683 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:04.893576 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:04.927421 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.190:39749
I20251023 08:00:04.927697 19526 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.190:39749 every 8 connection(s)
I20251023 08:00:04.931421 19527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:04.940491 19527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371: Bootstrap starting.
I20251023 08:00:04.944532 19527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:04.948257 19527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371: No bootstrap required, opened a new log
I20251023 08:00:04.950668 19527 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c6385fc0bf24edf9b1b27c711857371" member_type: VOTER }
I20251023 08:00:04.951054 19527 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:04.951287 19527 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c6385fc0bf24edf9b1b27c711857371, State: Initialized, Role: FOLLOWER
I20251023 08:00:04.951792 19527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c6385fc0bf24edf9b1b27c711857371" member_type: VOTER }
I20251023 08:00:04.952205 19527 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 08:00:04.952466 19527 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 08:00:04.952711 19527 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:04.956651 19527 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c6385fc0bf24edf9b1b27c711857371" member_type: VOTER }
I20251023 08:00:04.957182 19527 leader_election.cc:304] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c6385fc0bf24edf9b1b27c711857371; no voters: 
I20251023 08:00:04.958225 19527 leader_election.cc:290] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20251023 08:00:04.958631 19530 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:04.959882 19530 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371 [term 1 LEADER]: Becoming Leader. State: Replica: 7c6385fc0bf24edf9b1b27c711857371, State: Running, Role: LEADER
I20251023 08:00:04.960423 19530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c6385fc0bf24edf9b1b27c711857371" member_type: VOTER }
I20251023 08:00:04.960986 19527 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 08:00:04.962972 19532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c6385fc0bf24edf9b1b27c711857371. Latest consensus state: current_term: 1 leader_uuid: "7c6385fc0bf24edf9b1b27c711857371" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c6385fc0bf24edf9b1b27c711857371" member_type: VOTER } }
I20251023 08:00:04.962939 19531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c6385fc0bf24edf9b1b27c711857371" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c6385fc0bf24edf9b1b27c711857371" member_type: VOTER } }
I20251023 08:00:04.963575 19532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:04.963657 19531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:04.966643 19535 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 08:00:04.973448 19535 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 08:00:04.974738 18906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 08:00:04.981748 19535 catalog_manager.cc:1357] Generated new cluster ID: 693203145c29497d9f0ad6349f9b561d
I20251023 08:00:04.982012 19535 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 08:00:05.000633 19535 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 08:00:05.001848 19535 catalog_manager.cc:1514] Loading token signing keys...
I20251023 08:00:05.012794 19535 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371: Generated new TSK 0
I20251023 08:00:05.013398 19535 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 08:00:05.041253 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:05.046340 19548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:05.047484 19549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:05.048828 19551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:05.049576 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:05.050297 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:05.050467 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:05.050585 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206405050574 us; error 0 us; skew 500 ppm
I20251023 08:00:05.051049 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:05.053233 18906 webserver.cc:492] Webserver started at http://127.18.118.129:33797/ using document root <none> and password file <none>
I20251023 08:00:05.053637 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:05.053794 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:05.054016 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:05.054996 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1761206396810293-18906-0/minicluster-data/ts-0-root/instance:
uuid: "ae4e40320322462a8cbe73642158a863"
format_stamp: "Formatted at 2025-10-23 08:00:05 on dist-test-slave-wkn7"
I20251023 08:00:05.058992 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20251023 08:00:05.061893 19556 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:05.062595 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20251023 08:00:05.062846 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1761206396810293-18906-0/minicluster-data/ts-0-root
uuid: "ae4e40320322462a8cbe73642158a863"
format_stamp: "Formatted at 2025-10-23 08:00:05 on dist-test-slave-wkn7"
I20251023 08:00:05.063118 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1761206396810293-18906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1761206396810293-18906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1761206396810293-18906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:05.075230 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:05.076272 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:05.077729 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:00:05.080797 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:05.081018 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:05.081286 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:05.081441 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:05.121367 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.129:41011
I20251023 08:00:05.121454 19618 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.129:41011 every 8 connection(s)
I20251023 08:00:05.125687 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:05.132603 19623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:05.134366 19624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:05.135051 19626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:05.136377 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:05.137269 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:05.137519 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:05.137674 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206405137660 us; error 0 us; skew 500 ppm
I20251023 08:00:05.138340 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:05.138559 19619 heartbeater.cc:344] Connected to a master server at 127.18.118.190:39749
I20251023 08:00:05.138906 19619 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:05.139675 19619 heartbeater.cc:507] Master 127.18.118.190:39749 requested a full tablet report, sending...
I20251023 08:00:05.140753 18906 webserver.cc:492] Webserver started at http://127.18.118.130:44491/ using document root <none> and password file <none>
I20251023 08:00:05.141319 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:05.141561 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:05.141727 19492 ts_manager.cc:194] Registered new tserver with Master: ae4e40320322462a8cbe73642158a863 (127.18.118.129:41011)
I20251023 08:00:05.141886 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:05.143275 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1761206396810293-18906-0/minicluster-data/ts-1-root/instance:
uuid: "4c2b9c19abad43ea823b76d49a93027e"
format_stamp: "Formatted at 2025-10-23 08:00:05 on dist-test-slave-wkn7"
I20251023 08:00:05.143968 19492 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56256
I20251023 08:00:05.148288 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.002s
I20251023 08:00:05.151842 19631 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:05.152660 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20251023 08:00:05.152908 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1761206396810293-18906-0/minicluster-data/ts-1-root
uuid: "4c2b9c19abad43ea823b76d49a93027e"
format_stamp: "Formatted at 2025-10-23 08:00:05 on dist-test-slave-wkn7"
I20251023 08:00:05.153196 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1761206396810293-18906-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1761206396810293-18906-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1761206396810293-18906-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:05.183270 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:05.184177 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:05.185523 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:00:05.187556 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:05.187723 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:05.187958 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:05.188103 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:05.223651 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.130:40123
I20251023 08:00:05.223737 19693 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.130:40123 every 8 connection(s)
I20251023 08:00:05.236948 19694 heartbeater.cc:344] Connected to a master server at 127.18.118.190:39749
I20251023 08:00:05.237335 19694 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:05.238026 19694 heartbeater.cc:507] Master 127.18.118.190:39749 requested a full tablet report, sending...
I20251023 08:00:05.239778 19492 ts_manager.cc:194] Registered new tserver with Master: 4c2b9c19abad43ea823b76d49a93027e (127.18.118.130:40123)
I20251023 08:00:05.239905 18906 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.013297124s
I20251023 08:00:05.241044 19492 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56268
I20251023 08:00:05.246395 19619 heartbeater.cc:499] Master 127.18.118.190:39749 was elected leader, sending a full tablet report...
I20251023 08:00:05.263530 19492 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56270:
name: "test-table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20251023 08:00:05.264902 19492 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20251023 08:00:05.292970 19659 tablet_service.cc:1505] Processing CreateTablet for tablet 297a7306fa1c4834b8151a25d31d04c4 (DEFAULT_TABLE table=test-table [id=7386db9b110f4dc288004cfe88ff6cad]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 08:00:05.293972 19659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 297a7306fa1c4834b8151a25d31d04c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:05.294031 19584 tablet_service.cc:1505] Processing CreateTablet for tablet 297a7306fa1c4834b8151a25d31d04c4 (DEFAULT_TABLE table=test-table [id=7386db9b110f4dc288004cfe88ff6cad]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 08:00:05.294955 19584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 297a7306fa1c4834b8151a25d31d04c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:05.306303 19707 tablet_bootstrap.cc:492] T 297a7306fa1c4834b8151a25d31d04c4 P 4c2b9c19abad43ea823b76d49a93027e: Bootstrap starting.
I20251023 08:00:05.306699 19708 tablet_bootstrap.cc:492] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863: Bootstrap starting.
I20251023 08:00:05.311429 19707 tablet_bootstrap.cc:654] T 297a7306fa1c4834b8151a25d31d04c4 P 4c2b9c19abad43ea823b76d49a93027e: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:05.311547 19708 tablet_bootstrap.cc:654] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:05.315778 19707 tablet_bootstrap.cc:492] T 297a7306fa1c4834b8151a25d31d04c4 P 4c2b9c19abad43ea823b76d49a93027e: No bootstrap required, opened a new log
I20251023 08:00:05.315826 19708 tablet_bootstrap.cc:492] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863: No bootstrap required, opened a new log
I20251023 08:00:05.316321 19707 ts_tablet_manager.cc:1403] T 297a7306fa1c4834b8151a25d31d04c4 P 4c2b9c19abad43ea823b76d49a93027e: Time spent bootstrapping tablet: real 0.010s	user 0.004s	sys 0.005s
I20251023 08:00:05.316438 19708 ts_tablet_manager.cc:1403] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863: Time spent bootstrapping tablet: real 0.010s	user 0.004s	sys 0.005s
I20251023 08:00:05.318181 19707 raft_consensus.cc:359] T 297a7306fa1c4834b8151a25d31d04c4 P 4c2b9c19abad43ea823b76d49a93027e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2b9c19abad43ea823b76d49a93027e" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40123 } } peers { permanent_uuid: "ae4e40320322462a8cbe73642158a863" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41011 } }
I20251023 08:00:05.318326 19708 raft_consensus.cc:359] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2b9c19abad43ea823b76d49a93027e" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40123 } } peers { permanent_uuid: "ae4e40320322462a8cbe73642158a863" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41011 } }
I20251023 08:00:05.318622 19707 raft_consensus.cc:740] T 297a7306fa1c4834b8151a25d31d04c4 P 4c2b9c19abad43ea823b76d49a93027e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c2b9c19abad43ea823b76d49a93027e, State: Initialized, Role: FOLLOWER
I20251023 08:00:05.318816 19708 raft_consensus.cc:740] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae4e40320322462a8cbe73642158a863, State: Initialized, Role: FOLLOWER
I20251023 08:00:05.319116 19707 consensus_queue.cc:260] T 297a7306fa1c4834b8151a25d31d04c4 P 4c2b9c19abad43ea823b76d49a93027e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2b9c19abad43ea823b76d49a93027e" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40123 } } peers { permanent_uuid: "ae4e40320322462a8cbe73642158a863" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41011 } }
I20251023 08:00:05.319293 19708 consensus_queue.cc:260] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2b9c19abad43ea823b76d49a93027e" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40123 } } peers { permanent_uuid: "ae4e40320322462a8cbe73642158a863" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41011 } }
I20251023 08:00:05.321882 19694 heartbeater.cc:499] Master 127.18.118.190:39749 was elected leader, sending a full tablet report...
I20251023 08:00:05.322494 19707 ts_tablet_manager.cc:1434] T 297a7306fa1c4834b8151a25d31d04c4 P 4c2b9c19abad43ea823b76d49a93027e: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.003s
I20251023 08:00:05.324082 19708 ts_tablet_manager.cc:1434] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20251023 08:00:05.350803 18906 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20251023 08:00:05.351019 18906 raft_consensus.cc:423] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [term 0 FOLLOWER]: Emulating election...
I20251023 08:00:05.351197 18906 raft_consensus.cc:3060] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:05.355098 18906 raft_consensus.cc:697] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [term 1 LEADER]: Becoming Leader. State: Replica: ae4e40320322462a8cbe73642158a863, State: Running, Role: LEADER
W20251023 08:00:05.355453 19619 heartbeater.cc:646] Failed to heartbeat to 127.18.118.190:39749 (0 consecutive failures): IO error: failing all heartbeats for tests
I20251023 08:00:05.355626 18906 consensus_queue.cc:237] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2b9c19abad43ea823b76d49a93027e" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40123 } } peers { permanent_uuid: "ae4e40320322462a8cbe73642158a863" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41011 } }
I20251023 08:00:05.358258 18906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20251023 08:00:05.375921 18906 cluster_itest_util.cc:257] Not converged past 1 yet: 1.1 0.0
I20251023 08:00:05.479864 18906 cluster_itest_util.cc:257] Not converged past 1 yet: 1.1 0.0
I20251023 08:00:05.683919 18906 cluster_itest_util.cc:257] Not converged past 1 yet: 1.1 0.0
I20251023 08:00:05.821933 19669 raft_consensus.cc:3060] T 297a7306fa1c4834b8151a25d31d04c4 P 4c2b9c19abad43ea823b76d49a93027e [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:05.827638 19712 consensus_queue.cc:1048] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2b9c19abad43ea823b76d49a93027e" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 08:00:05.988791 18906 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20251023 08:00:05.989008 18906 raft_consensus.cc:423] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [term 1 LEADER]: Emulating election...
I20251023 08:00:05.989288 18906 raft_consensus.cc:3055] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 08:00:05.989506 18906 raft_consensus.cc:740] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ae4e40320322462a8cbe73642158a863, State: Running, Role: LEADER
I20251023 08:00:05.989945 18906 consensus_queue.cc:260] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2b9c19abad43ea823b76d49a93027e" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40123 } } peers { permanent_uuid: "ae4e40320322462a8cbe73642158a863" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41011 } }
I20251023 08:00:05.990553 18906 raft_consensus.cc:3060] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:05.994439 18906 raft_consensus.cc:697] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [term 2 LEADER]: Becoming Leader. State: Replica: ae4e40320322462a8cbe73642158a863, State: Running, Role: LEADER
I20251023 08:00:05.994935 18906 consensus_queue.cc:237] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2b9c19abad43ea823b76d49a93027e" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40123 } } peers { permanent_uuid: "ae4e40320322462a8cbe73642158a863" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41011 } }
I20251023 08:00:05.996110 18906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20251023 08:00:05.999290 18906 cluster_itest_util.cc:257] Not converged past 2 yet: 2.2 1.1
I20251023 08:00:06.103152 18906 cluster_itest_util.cc:257] Not converged past 2 yet: 2.2 1.1
I20251023 08:00:06.307265 18906 cluster_itest_util.cc:257] Not converged past 2 yet: 2.2 1.1
I20251023 08:00:06.524992 19669 raft_consensus.cc:3060] T 297a7306fa1c4834b8151a25d31d04c4 P 4c2b9c19abad43ea823b76d49a93027e [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:06.529506 19669 raft_consensus.cc:1275] T 297a7306fa1c4834b8151a25d31d04c4 P 4c2b9c19abad43ea823b76d49a93027e [term 2 FOLLOWER]: Refusing update from remote peer ae4e40320322462a8cbe73642158a863: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 08:00:06.530587 19721 consensus_queue.cc:1048] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2b9c19abad43ea823b76d49a93027e" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 08:00:06.612537 18906 ts_tablet_manager-itest.cc:368] Electing peer 0...
I20251023 08:00:06.612756 18906 raft_consensus.cc:423] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [term 2 LEADER]: Emulating election...
I20251023 08:00:06.612980 18906 raft_consensus.cc:3055] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [term 2 LEADER]: Stepping down as leader of term 2
I20251023 08:00:06.613188 18906 raft_consensus.cc:740] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: ae4e40320322462a8cbe73642158a863, State: Running, Role: LEADER
I20251023 08:00:06.613620 18906 consensus_queue.cc:260] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2b9c19abad43ea823b76d49a93027e" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40123 } } peers { permanent_uuid: "ae4e40320322462a8cbe73642158a863" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41011 } }
I20251023 08:00:06.614229 18906 raft_consensus.cc:3060] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [term 2 FOLLOWER]: Advancing to term 3
I20251023 08:00:06.618113 18906 raft_consensus.cc:697] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [term 3 LEADER]: Becoming Leader. State: Replica: ae4e40320322462a8cbe73642158a863, State: Running, Role: LEADER
I20251023 08:00:06.618626 18906 consensus_queue.cc:237] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4c2b9c19abad43ea823b76d49a93027e" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40123 } } peers { permanent_uuid: "ae4e40320322462a8cbe73642158a863" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41011 } }
I20251023 08:00:06.619838 18906 ts_tablet_manager-itest.cc:371] Waiting for servers to agree...
I20251023 08:00:06.623152 18906 cluster_itest_util.cc:257] Not converged past 3 yet: 3.3 2.2
I20251023 08:00:06.726866 18906 cluster_itest_util.cc:257] Not converged past 3 yet: 3.3 2.2
I20251023 08:00:06.931361 18906 cluster_itest_util.cc:257] Not converged past 3 yet: 3.3 2.2
I20251023 08:00:07.228717 19669 raft_consensus.cc:3060] T 297a7306fa1c4834b8151a25d31d04c4 P 4c2b9c19abad43ea823b76d49a93027e [term 2 FOLLOWER]: Advancing to term 3
I20251023 08:00:07.237811 19669 raft_consensus.cc:1275] T 297a7306fa1c4834b8151a25d31d04c4 P 4c2b9c19abad43ea823b76d49a93027e [term 3 FOLLOWER]: Refusing update from remote peer ae4e40320322462a8cbe73642158a863: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20251023 08:00:07.239513 18906 cluster_itest_util.cc:257] Not converged past 3 yet: 3.3 2.2
I20251023 08:00:07.239708 19727 consensus_queue.cc:1048] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c2b9c19abad43ea823b76d49a93027e" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20251023 08:00:07.647698 18906 tablet_server.cc:178] TabletServer@127.18.118.129:0 shutting down...
I20251023 08:00:07.670145 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:07.671212 18906 tablet_replica.cc:333] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863: stopping tablet replica
I20251023 08:00:07.671871 18906 raft_consensus.cc:2243] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [term 3 LEADER]: Raft consensus shutting down.
I20251023 08:00:07.672485 18906 raft_consensus.cc:2272] T 297a7306fa1c4834b8151a25d31d04c4 P ae4e40320322462a8cbe73642158a863 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:07.681350 18906 tablet_server.cc:195] TabletServer@127.18.118.129:0 shutdown complete.
I20251023 08:00:07.690488 18906 tablet_server.cc:178] TabletServer@127.18.118.130:0 shutting down...
I20251023 08:00:07.709527 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:07.710445 18906 tablet_replica.cc:333] T 297a7306fa1c4834b8151a25d31d04c4 P 4c2b9c19abad43ea823b76d49a93027e: stopping tablet replica
I20251023 08:00:07.711102 18906 raft_consensus.cc:2243] T 297a7306fa1c4834b8151a25d31d04c4 P 4c2b9c19abad43ea823b76d49a93027e [term 3 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:07.711568 18906 raft_consensus.cc:2272] T 297a7306fa1c4834b8151a25d31d04c4 P 4c2b9c19abad43ea823b76d49a93027e [term 3 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:07.730911 18906 tablet_server.cc:195] TabletServer@127.18.118.130:0 shutdown complete.
I20251023 08:00:07.739773 18906 master.cc:561] Master@127.18.118.190:39749 shutting down...
I20251023 08:00:07.756271 18906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371 [term 1 LEADER]: Raft consensus shutting down.
I20251023 08:00:07.756850 18906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:07.757174 18906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 7c6385fc0bf24edf9b1b27c711857371: stopping tablet replica
I20251023 08:00:07.775151 18906 master.cc:583] Master@127.18.118.190:39749 shutdown complete.
[       OK ] LeadershipChangeReportingTest.TestUpdatedConsensusState (2952 ms)
[----------] 3 tests from LeadershipChangeReportingTest (10830 ms total)

[----------] 4 tests from TsTabletManagerITest
[ RUN      ] TsTabletManagerITest.ReportOnReplicaHealthStatus
I20251023 08:00:07.806598 18906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.18.118.190:33191
I20251023 08:00:07.807677 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:07.813555 19736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:07.814978 19737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:07.816417 19739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:07.817497 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:07.818225 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:07.818420 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:07.818574 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206407818555 us; error 0 us; skew 500 ppm
I20251023 08:00:07.819064 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:07.821525 18906 webserver.cc:492] Webserver started at http://127.18.118.190:36895/ using document root <none> and password file <none>
I20251023 08:00:07.822018 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:07.822203 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:07.822466 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:07.823589 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1761206396810293-18906-0/minicluster-data/master-0-root/instance:
uuid: "799c8cc7af614fa4af51b58e80f1573f"
format_stamp: "Formatted at 2025-10-23 08:00:07 on dist-test-slave-wkn7"
I20251023 08:00:07.828024 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20251023 08:00:07.831667 19744 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:07.832391 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20251023 08:00:07.832672 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1761206396810293-18906-0/minicluster-data/master-0-root
uuid: "799c8cc7af614fa4af51b58e80f1573f"
format_stamp: "Formatted at 2025-10-23 08:00:07 on dist-test-slave-wkn7"
I20251023 08:00:07.832948 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1761206396810293-18906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1761206396810293-18906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1761206396810293-18906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:07.847828 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:07.849048 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:07.888314 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.190:33191
I20251023 08:00:07.888423 19795 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.190:33191 every 8 connection(s)
I20251023 08:00:07.892048 19796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:07.902158 19796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f: Bootstrap starting.
I20251023 08:00:07.906883 19796 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:07.911006 19796 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f: No bootstrap required, opened a new log
I20251023 08:00:07.912972 19796 raft_consensus.cc:359] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "799c8cc7af614fa4af51b58e80f1573f" member_type: VOTER }
I20251023 08:00:07.913408 19796 raft_consensus.cc:385] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:07.913651 19796 raft_consensus.cc:740] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 799c8cc7af614fa4af51b58e80f1573f, State: Initialized, Role: FOLLOWER
I20251023 08:00:07.914211 19796 consensus_queue.cc:260] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "799c8cc7af614fa4af51b58e80f1573f" member_type: VOTER }
I20251023 08:00:07.914641 19796 raft_consensus.cc:399] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 08:00:07.914866 19796 raft_consensus.cc:493] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 08:00:07.915112 19796 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:07.919243 19796 raft_consensus.cc:515] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "799c8cc7af614fa4af51b58e80f1573f" member_type: VOTER }
I20251023 08:00:07.919740 19796 leader_election.cc:304] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 799c8cc7af614fa4af51b58e80f1573f; no voters: 
I20251023 08:00:07.920853 19796 leader_election.cc:290] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20251023 08:00:07.921258 19799 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:07.922565 19799 raft_consensus.cc:697] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f [term 1 LEADER]: Becoming Leader. State: Replica: 799c8cc7af614fa4af51b58e80f1573f, State: Running, Role: LEADER
I20251023 08:00:07.923143 19799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "799c8cc7af614fa4af51b58e80f1573f" member_type: VOTER }
I20251023 08:00:07.923722 19796 sys_catalog.cc:565] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f [sys.catalog]: configured and running, proceeding with master startup.
I20251023 08:00:07.925868 19801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 799c8cc7af614fa4af51b58e80f1573f. Latest consensus state: current_term: 1 leader_uuid: "799c8cc7af614fa4af51b58e80f1573f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "799c8cc7af614fa4af51b58e80f1573f" member_type: VOTER } }
I20251023 08:00:07.925874 19800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "799c8cc7af614fa4af51b58e80f1573f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "799c8cc7af614fa4af51b58e80f1573f" member_type: VOTER } }
I20251023 08:00:07.926524 19801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:07.926666 19800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:07.938903 18906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
W20251023 08:00:07.941984 19815 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251023 08:00:07.942245 19815 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251023 08:00:07.942544 19804 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 08:00:07.947108 19804 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 08:00:07.955428 19804 catalog_manager.cc:1357] Generated new cluster ID: 7e6945f7580d479a94fdf3e9a2e4a500
I20251023 08:00:07.955763 19804 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 08:00:07.983228 19804 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 08:00:07.984606 19804 catalog_manager.cc:1514] Loading token signing keys...
I20251023 08:00:07.997229 19804 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f: Generated new TSK 0
I20251023 08:00:07.997817 19804 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 08:00:08.005321 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:08.011236 19817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:08.011766 19818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:08.013372 19820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:08.014061 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:08.015055 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:08.015259 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:08.015383 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206408015371 us; error 0 us; skew 500 ppm
I20251023 08:00:08.015883 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:08.018386 18906 webserver.cc:492] Webserver started at http://127.18.118.129:46489/ using document root <none> and password file <none>
I20251023 08:00:08.018836 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:08.019004 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:08.019223 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:08.020311 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1761206396810293-18906-0/minicluster-data/ts-0-root/instance:
uuid: "7edf43e91f0f4140b6ce8afcd6e917a9"
format_stamp: "Formatted at 2025-10-23 08:00:08 on dist-test-slave-wkn7"
I20251023 08:00:08.024744 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20251023 08:00:08.028198 19825 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:08.028949 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20251023 08:00:08.029245 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1761206396810293-18906-0/minicluster-data/ts-0-root
uuid: "7edf43e91f0f4140b6ce8afcd6e917a9"
format_stamp: "Formatted at 2025-10-23 08:00:08 on dist-test-slave-wkn7"
I20251023 08:00:08.029531 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1761206396810293-18906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1761206396810293-18906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1761206396810293-18906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:08.061311 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:08.062463 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:08.063884 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:00:08.066093 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:08.066285 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:08.066530 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:08.066697 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:08.105865 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.129:38755
I20251023 08:00:08.105971 19887 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.129:38755 every 8 connection(s)
I20251023 08:00:08.110538 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:08.117655 19892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:08.119009 19893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:08.122053 18906 server_base.cc:1047] running on GCE node
W20251023 08:00:08.122763 19895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:08.123785 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:08.123994 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:08.124208 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206408124187 us; error 0 us; skew 500 ppm
I20251023 08:00:08.124313 19888 heartbeater.cc:344] Connected to a master server at 127.18.118.190:33191
I20251023 08:00:08.124672 19888 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:08.124784 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:08.125543 19888 heartbeater.cc:507] Master 127.18.118.190:33191 requested a full tablet report, sending...
I20251023 08:00:08.127365 18906 webserver.cc:492] Webserver started at http://127.18.118.130:46389/ using document root <none> and password file <none>
I20251023 08:00:08.127864 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:08.128077 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:08.127861 19761 ts_manager.cc:194] Registered new tserver with Master: 7edf43e91f0f4140b6ce8afcd6e917a9 (127.18.118.129:38755)
I20251023 08:00:08.128376 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:08.129594 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1761206396810293-18906-0/minicluster-data/ts-1-root/instance:
uuid: "d8aecc2473ce4e8a8f9f7070c958a49d"
format_stamp: "Formatted at 2025-10-23 08:00:08 on dist-test-slave-wkn7"
I20251023 08:00:08.129889 19761 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33730
I20251023 08:00:08.134203 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20251023 08:00:08.137634 19900 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:08.138411 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.002s
I20251023 08:00:08.138654 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1761206396810293-18906-0/minicluster-data/ts-1-root
uuid: "d8aecc2473ce4e8a8f9f7070c958a49d"
format_stamp: "Formatted at 2025-10-23 08:00:08 on dist-test-slave-wkn7"
I20251023 08:00:08.138897 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1761206396810293-18906-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1761206396810293-18906-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1761206396810293-18906-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:08.179627 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:08.180749 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:08.182142 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:00:08.184252 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:08.184460 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:08.184741 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:08.185038 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20251023 08:00:08.224303 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.130:40541
I20251023 08:00:08.224406 19962 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.130:40541 every 8 connection(s)
I20251023 08:00:08.228869 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:08.235033 19966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:08.235716 19967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:08.239154 19969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:08.239882 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:08.241024 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:08.241297 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:08.241487 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206408241469 us; error 0 us; skew 500 ppm
I20251023 08:00:08.241680 19963 heartbeater.cc:344] Connected to a master server at 127.18.118.190:33191
I20251023 08:00:08.242084 19963 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:08.242173 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:08.242996 19963 heartbeater.cc:507] Master 127.18.118.190:33191 requested a full tablet report, sending...
I20251023 08:00:08.245031 18906 webserver.cc:492] Webserver started at http://127.18.118.131:44199/ using document root <none> and password file <none>
I20251023 08:00:08.245397 19761 ts_manager.cc:194] Registered new tserver with Master: d8aecc2473ce4e8a8f9f7070c958a49d (127.18.118.130:40541)
I20251023 08:00:08.245703 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:08.246063 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:08.246719 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:08.247563 19761 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33740
I20251023 08:00:08.248344 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1761206396810293-18906-0/minicluster-data/ts-2-root/instance:
uuid: "a998d3abbe6c4b259db3e34afb2db643"
format_stamp: "Formatted at 2025-10-23 08:00:08 on dist-test-slave-wkn7"
I20251023 08:00:08.252988 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.004s
I20251023 08:00:08.256228 19974 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:08.256940 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20251023 08:00:08.257241 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1761206396810293-18906-0/minicluster-data/ts-2-root
uuid: "a998d3abbe6c4b259db3e34afb2db643"
format_stamp: "Formatted at 2025-10-23 08:00:08 on dist-test-slave-wkn7"
I20251023 08:00:08.257510 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1761206396810293-18906-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1761206396810293-18906-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1761206396810293-18906-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:08.283099 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:08.284276 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:08.285750 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:00:08.287864 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:08.288061 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:08.288282 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:08.288436 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:08.361969 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.131:33173
I20251023 08:00:08.362076 20036 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.131:33173 every 8 connection(s)
I20251023 08:00:08.377234 20037 heartbeater.cc:344] Connected to a master server at 127.18.118.190:33191
I20251023 08:00:08.377570 20037 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:08.378247 20037 heartbeater.cc:507] Master 127.18.118.190:33191 requested a full tablet report, sending...
I20251023 08:00:08.380066 19761 ts_manager.cc:194] Registered new tserver with Master: a998d3abbe6c4b259db3e34afb2db643 (127.18.118.131:33173)
I20251023 08:00:08.380409 18906 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013956525s
I20251023 08:00:08.381733 19761 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33750
I20251023 08:00:08.405901 19761 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33752:
name: "test-table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
  }
}
W20251023 08:00:08.407598 19761 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 08:00:08.453583 20002 tablet_service.cc:1505] Processing CreateTablet for tablet 02242dbee0b1425d8782aaf4543ffb4c (DEFAULT_TABLE table=test-table [id=eb220fac03a4403b8a88b51947c03c68]), partition=
I20251023 08:00:08.454442 19853 tablet_service.cc:1505] Processing CreateTablet for tablet 02242dbee0b1425d8782aaf4543ffb4c (DEFAULT_TABLE table=test-table [id=eb220fac03a4403b8a88b51947c03c68]), partition=
I20251023 08:00:08.454790 20002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02242dbee0b1425d8782aaf4543ffb4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:08.455540 19853 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02242dbee0b1425d8782aaf4543ffb4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:08.470331 19928 tablet_service.cc:1505] Processing CreateTablet for tablet 02242dbee0b1425d8782aaf4543ffb4c (DEFAULT_TABLE table=test-table [id=eb220fac03a4403b8a88b51947c03c68]), partition=
I20251023 08:00:08.471560 19928 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 02242dbee0b1425d8782aaf4543ffb4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:08.471779 20053 tablet_bootstrap.cc:492] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9: Bootstrap starting.
I20251023 08:00:08.475152 20052 tablet_bootstrap.cc:492] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643: Bootstrap starting.
I20251023 08:00:08.479609 20053 tablet_bootstrap.cc:654] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:08.483577 20052 tablet_bootstrap.cc:654] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:08.485952 20053 tablet_bootstrap.cc:492] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9: No bootstrap required, opened a new log
I20251023 08:00:08.486407 20053 ts_tablet_manager.cc:1403] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9: Time spent bootstrapping tablet: real 0.015s	user 0.012s	sys 0.000s
I20251023 08:00:08.489387 20052 tablet_bootstrap.cc:492] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643: No bootstrap required, opened a new log
I20251023 08:00:08.490082 20052 ts_tablet_manager.cc:1403] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643: Time spent bootstrapping tablet: real 0.015s	user 0.007s	sys 0.006s
I20251023 08:00:08.491461 20056 tablet_bootstrap.cc:492] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d: Bootstrap starting.
I20251023 08:00:08.489372 20053 raft_consensus.cc:359] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998d3abbe6c4b259db3e34afb2db643" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 33173 } } peers { permanent_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38755 } } peers { permanent_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40541 } }
I20251023 08:00:08.492113 20053 raft_consensus.cc:385] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:08.492501 20053 raft_consensus.cc:740] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7edf43e91f0f4140b6ce8afcd6e917a9, State: Initialized, Role: FOLLOWER
I20251023 08:00:08.493227 20053 consensus_queue.cc:260] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998d3abbe6c4b259db3e34afb2db643" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 33173 } } peers { permanent_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38755 } } peers { permanent_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40541 } }
I20251023 08:00:08.493600 20052 raft_consensus.cc:359] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998d3abbe6c4b259db3e34afb2db643" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 33173 } } peers { permanent_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38755 } } peers { permanent_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40541 } }
I20251023 08:00:08.494444 20052 raft_consensus.cc:385] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:08.494856 20052 raft_consensus.cc:740] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a998d3abbe6c4b259db3e34afb2db643, State: Initialized, Role: FOLLOWER
I20251023 08:00:08.496240 19888 heartbeater.cc:499] Master 127.18.118.190:33191 was elected leader, sending a full tablet report...
I20251023 08:00:08.496677 20053 ts_tablet_manager.cc:1434] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20251023 08:00:08.495791 20052 consensus_queue.cc:260] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998d3abbe6c4b259db3e34afb2db643" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 33173 } } peers { permanent_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38755 } } peers { permanent_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40541 } }
I20251023 08:00:08.498656 20056 tablet_bootstrap.cc:654] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:08.499905 20037 heartbeater.cc:499] Master 127.18.118.190:33191 was elected leader, sending a full tablet report...
I20251023 08:00:08.500571 20052 ts_tablet_manager.cc:1434] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20251023 08:00:08.504992 20056 tablet_bootstrap.cc:492] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d: No bootstrap required, opened a new log
I20251023 08:00:08.505543 20056 ts_tablet_manager.cc:1403] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d: Time spent bootstrapping tablet: real 0.014s	user 0.013s	sys 0.000s
I20251023 08:00:08.508580 20056 raft_consensus.cc:359] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998d3abbe6c4b259db3e34afb2db643" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 33173 } } peers { permanent_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38755 } } peers { permanent_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40541 } }
I20251023 08:00:08.509295 20056 raft_consensus.cc:385] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:08.509603 20056 raft_consensus.cc:740] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8aecc2473ce4e8a8f9f7070c958a49d, State: Initialized, Role: FOLLOWER
I20251023 08:00:08.510280 20056 consensus_queue.cc:260] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998d3abbe6c4b259db3e34afb2db643" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 33173 } } peers { permanent_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38755 } } peers { permanent_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40541 } }
I20251023 08:00:08.512993 19963 heartbeater.cc:499] Master 127.18.118.190:33191 was elected leader, sending a full tablet report...
I20251023 08:00:08.513370 20056 ts_tablet_manager.cc:1434] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20251023 08:00:08.664701 20057 raft_consensus.cc:493] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 08:00:08.665216 20057 raft_consensus.cc:515] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998d3abbe6c4b259db3e34afb2db643" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 33173 } } peers { permanent_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38755 } } peers { permanent_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40541 } }
I20251023 08:00:08.667255 20057 leader_election.cc:290] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a998d3abbe6c4b259db3e34afb2db643 (127.18.118.131:33173), d8aecc2473ce4e8a8f9f7070c958a49d (127.18.118.130:40541)
I20251023 08:00:08.677410 19938 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02242dbee0b1425d8782aaf4543ffb4c" candidate_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" is_pre_election: true
I20251023 08:00:08.677732 20012 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02242dbee0b1425d8782aaf4543ffb4c" candidate_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a998d3abbe6c4b259db3e34afb2db643" is_pre_election: true
I20251023 08:00:08.678131 19938 raft_consensus.cc:2468] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7edf43e91f0f4140b6ce8afcd6e917a9 in term 0.
I20251023 08:00:08.678309 20012 raft_consensus.cc:2468] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7edf43e91f0f4140b6ce8afcd6e917a9 in term 0.
I20251023 08:00:08.679348 19826 leader_election.cc:304] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edf43e91f0f4140b6ce8afcd6e917a9, d8aecc2473ce4e8a8f9f7070c958a49d; no voters: 
I20251023 08:00:08.680028 20057 raft_consensus.cc:2804] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 08:00:08.680306 20057 raft_consensus.cc:493] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 08:00:08.680547 20057 raft_consensus.cc:3060] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:08.685312 20057 raft_consensus.cc:515] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998d3abbe6c4b259db3e34afb2db643" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 33173 } } peers { permanent_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38755 } } peers { permanent_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40541 } }
I20251023 08:00:08.687003 20057 leader_election.cc:290] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [CANDIDATE]: Term 1 election: Requested vote from peers a998d3abbe6c4b259db3e34afb2db643 (127.18.118.131:33173), d8aecc2473ce4e8a8f9f7070c958a49d (127.18.118.130:40541)
I20251023 08:00:08.687669 20012 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02242dbee0b1425d8782aaf4543ffb4c" candidate_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a998d3abbe6c4b259db3e34afb2db643"
I20251023 08:00:08.687932 19938 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02242dbee0b1425d8782aaf4543ffb4c" candidate_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d"
I20251023 08:00:08.688113 20012 raft_consensus.cc:3060] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:08.688385 19938 raft_consensus.cc:3060] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:08.692827 20012 raft_consensus.cc:2468] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7edf43e91f0f4140b6ce8afcd6e917a9 in term 1.
I20251023 08:00:08.693071 19938 raft_consensus.cc:2468] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7edf43e91f0f4140b6ce8afcd6e917a9 in term 1.
I20251023 08:00:08.693954 19827 leader_election.cc:304] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7edf43e91f0f4140b6ce8afcd6e917a9, a998d3abbe6c4b259db3e34afb2db643; no voters: 
I20251023 08:00:08.694706 20057 raft_consensus.cc:2804] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:08.695497 20057 raft_consensus.cc:697] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [term 1 LEADER]: Becoming Leader. State: Replica: 7edf43e91f0f4140b6ce8afcd6e917a9, State: Running, Role: LEADER
I20251023 08:00:08.696125 20057 consensus_queue.cc:237] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998d3abbe6c4b259db3e34afb2db643" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 33173 } } peers { permanent_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38755 } } peers { permanent_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40541 } }
I20251023 08:00:08.702795 19761 catalog_manager.cc:5649] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7edf43e91f0f4140b6ce8afcd6e917a9 (127.18.118.129). New cstate: current_term: 1 leader_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998d3abbe6c4b259db3e34afb2db643" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 33173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40541 } health_report { overall_health: UNKNOWN } } }
I20251023 08:00:09.094347 20057 consensus_queue.cc:1048] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a998d3abbe6c4b259db3e34afb2db643" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 33173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 08:00:09.104628 20057 consensus_queue.cc:1048] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40541 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 08:00:09.792207 18906 tablet_replica.cc:333] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9: stopping tablet replica
I20251023 08:00:09.792762 18906 raft_consensus.cc:2243] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [term 1 LEADER]: Raft consensus shutting down.
I20251023 08:00:09.793603 18906 raft_consensus.cc:2272] T 02242dbee0b1425d8782aaf4543ffb4c P 7edf43e91f0f4140b6ce8afcd6e917a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:10.986714 20076 raft_consensus.cc:493] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7edf43e91f0f4140b6ce8afcd6e917a9)
I20251023 08:00:10.987192 20076 raft_consensus.cc:515] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998d3abbe6c4b259db3e34afb2db643" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 33173 } } peers { permanent_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38755 } } peers { permanent_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40541 } }
I20251023 08:00:10.988960 20076 leader_election.cc:290] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7edf43e91f0f4140b6ce8afcd6e917a9 (127.18.118.129:38755), d8aecc2473ce4e8a8f9f7070c958a49d (127.18.118.130:40541)
I20251023 08:00:11.013278 19863 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02242dbee0b1425d8782aaf4543ffb4c" candidate_uuid: "a998d3abbe6c4b259db3e34afb2db643" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" is_pre_election: true
I20251023 08:00:11.014070 19938 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02242dbee0b1425d8782aaf4543ffb4c" candidate_uuid: "a998d3abbe6c4b259db3e34afb2db643" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" is_pre_election: true
W20251023 08:00:11.014835 19978 leader_election.cc:343] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7edf43e91f0f4140b6ce8afcd6e917a9 (127.18.118.129:38755): Illegal state: cannot vote while shut down
I20251023 08:00:11.015779 19975 leader_election.cc:304] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a998d3abbe6c4b259db3e34afb2db643; no voters: 7edf43e91f0f4140b6ce8afcd6e917a9, d8aecc2473ce4e8a8f9f7070c958a49d
I20251023 08:00:11.016475 20076 raft_consensus.cc:2749] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251023 08:00:11.204536 20081 raft_consensus.cc:493] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7edf43e91f0f4140b6ce8afcd6e917a9)
I20251023 08:00:11.205017 20081 raft_consensus.cc:515] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998d3abbe6c4b259db3e34afb2db643" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 33173 } } peers { permanent_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38755 } } peers { permanent_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40541 } }
I20251023 08:00:11.207039 20081 leader_election.cc:290] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a998d3abbe6c4b259db3e34afb2db643 (127.18.118.131:33173), 7edf43e91f0f4140b6ce8afcd6e917a9 (127.18.118.129:38755)
I20251023 08:00:11.218467 19863 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02242dbee0b1425d8782aaf4543ffb4c" candidate_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" is_pre_election: true
W20251023 08:00:11.219861 19904 leader_election.cc:343] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7edf43e91f0f4140b6ce8afcd6e917a9 (127.18.118.129:38755): Illegal state: cannot vote while shut down
I20251023 08:00:11.220352 20012 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02242dbee0b1425d8782aaf4543ffb4c" candidate_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a998d3abbe6c4b259db3e34afb2db643" is_pre_election: true
I20251023 08:00:11.220801 20012 raft_consensus.cc:2468] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8aecc2473ce4e8a8f9f7070c958a49d in term 1.
I20251023 08:00:11.221865 19902 leader_election.cc:304] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a998d3abbe6c4b259db3e34afb2db643, d8aecc2473ce4e8a8f9f7070c958a49d; no voters: 7edf43e91f0f4140b6ce8afcd6e917a9
I20251023 08:00:11.222527 20081 raft_consensus.cc:2804] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 08:00:11.222813 20081 raft_consensus.cc:493] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7edf43e91f0f4140b6ce8afcd6e917a9)
I20251023 08:00:11.223096 20081 raft_consensus.cc:3060] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:11.227737 20081 raft_consensus.cc:515] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998d3abbe6c4b259db3e34afb2db643" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 33173 } } peers { permanent_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38755 } } peers { permanent_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40541 } }
I20251023 08:00:11.229231 20081 leader_election.cc:290] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [CANDIDATE]: Term 2 election: Requested vote from peers a998d3abbe6c4b259db3e34afb2db643 (127.18.118.131:33173), 7edf43e91f0f4140b6ce8afcd6e917a9 (127.18.118.129:38755)
I20251023 08:00:11.230471 20012 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02242dbee0b1425d8782aaf4543ffb4c" candidate_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a998d3abbe6c4b259db3e34afb2db643"
I20251023 08:00:11.230506 19863 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "02242dbee0b1425d8782aaf4543ffb4c" candidate_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9"
I20251023 08:00:11.231302 20012 raft_consensus.cc:3060] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643 [term 1 FOLLOWER]: Advancing to term 2
W20251023 08:00:11.231997 19904 leader_election.cc:343] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 7edf43e91f0f4140b6ce8afcd6e917a9 (127.18.118.129:38755): Illegal state: cannot vote while shut down
I20251023 08:00:11.236443 20012 raft_consensus.cc:2468] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8aecc2473ce4e8a8f9f7070c958a49d in term 2.
I20251023 08:00:11.237402 19902 leader_election.cc:304] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a998d3abbe6c4b259db3e34afb2db643, d8aecc2473ce4e8a8f9f7070c958a49d; no voters: 7edf43e91f0f4140b6ce8afcd6e917a9
I20251023 08:00:11.238054 20081 raft_consensus.cc:2804] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [term 2 FOLLOWER]: Leader election won for term 2
I20251023 08:00:11.239118 20081 raft_consensus.cc:697] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [term 2 LEADER]: Becoming Leader. State: Replica: d8aecc2473ce4e8a8f9f7070c958a49d, State: Running, Role: LEADER
I20251023 08:00:11.239779 20081 consensus_queue.cc:237] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a998d3abbe6c4b259db3e34afb2db643" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 33173 } } peers { permanent_uuid: "7edf43e91f0f4140b6ce8afcd6e917a9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38755 } } peers { permanent_uuid: "d8aecc2473ce4e8a8f9f7070c958a49d" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40541 } }
I20251023 08:00:11.713039 20012 raft_consensus.cc:1275] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643 [term 2 FOLLOWER]: Refusing update from remote peer d8aecc2473ce4e8a8f9f7070c958a49d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 08:00:11.714661 20081 consensus_queue.cc:1048] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a998d3abbe6c4b259db3e34afb2db643" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 33173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 08:00:11.725510 19904 consensus_queue.cc:579] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [LEADER]: The tablet replica hosted on peer 7edf43e91f0f4140b6ce8afcd6e917a9 has failed
W20251023 08:00:11.725999 19904 consensus_peers.cc:597] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d -> Peer 7edf43e91f0f4140b6ce8afcd6e917a9 (127.18.118.129:38755): Couldn't send request to peer 7edf43e91f0f4140b6ce8afcd6e917a9. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20251023 08:00:11.828531 18906 tablet_server.cc:178] TabletServer@127.18.118.129:0 shutting down...
I20251023 08:00:11.845407 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:11.862150 18906 tablet_server.cc:195] TabletServer@127.18.118.129:0 shutdown complete.
I20251023 08:00:11.870036 18906 tablet_server.cc:178] TabletServer@127.18.118.130:0 shutting down...
I20251023 08:00:11.885979 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:11.886585 18906 tablet_replica.cc:333] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d: stopping tablet replica
I20251023 08:00:11.887055 18906 raft_consensus.cc:2243] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [term 2 LEADER]: Raft consensus shutting down.
I20251023 08:00:11.887918 18906 raft_consensus.cc:2272] T 02242dbee0b1425d8782aaf4543ffb4c P d8aecc2473ce4e8a8f9f7070c958a49d [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:11.896261 18906 tablet_server.cc:195] TabletServer@127.18.118.130:0 shutdown complete.
I20251023 08:00:11.903403 18906 tablet_server.cc:178] TabletServer@127.18.118.131:0 shutting down...
I20251023 08:00:11.917799 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:11.918404 18906 tablet_replica.cc:333] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643: stopping tablet replica
I20251023 08:00:11.918902 18906 raft_consensus.cc:2243] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:11.919298 18906 raft_consensus.cc:2272] T 02242dbee0b1425d8782aaf4543ffb4c P a998d3abbe6c4b259db3e34afb2db643 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:11.927078 18906 tablet_server.cc:195] TabletServer@127.18.118.131:0 shutdown complete.
I20251023 08:00:11.934832 18906 master.cc:561] Master@127.18.118.190:33191 shutting down...
I20251023 08:00:11.948407 18906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f [term 1 LEADER]: Raft consensus shutting down.
I20251023 08:00:11.948880 18906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:11.949208 18906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 799c8cc7af614fa4af51b58e80f1573f: stopping tablet replica
I20251023 08:00:11.965976 18906 master.cc:583] Master@127.18.118.190:33191 shutdown complete.
[       OK ] TsTabletManagerITest.ReportOnReplicaHealthStatus (4184 ms)
[ RUN      ] TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters
I20251023 08:00:11.990307 18906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.18.118.190:42281
I20251023 08:00:11.991322 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:11.996080 20096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:11.996897 20097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:11.998286 20099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:11.998732 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:11.999394 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:11.999563 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:11.999701 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206411999687 us; error 0 us; skew 500 ppm
I20251023 08:00:12.000209 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:12.002417 18906 webserver.cc:492] Webserver started at http://127.18.118.190:44405/ using document root <none> and password file <none>
I20251023 08:00:12.002877 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:12.003046 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:12.003300 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:12.004299 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1761206396810293-18906-0/minicluster-data/master-0-root/instance:
uuid: "b9085d66147148448f2c007b78201139"
format_stamp: "Formatted at 2025-10-23 08:00:11 on dist-test-slave-wkn7"
I20251023 08:00:12.008415 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20251023 08:00:12.011565 20104 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:12.012275 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20251023 08:00:12.012526 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1761206396810293-18906-0/minicluster-data/master-0-root
uuid: "b9085d66147148448f2c007b78201139"
format_stamp: "Formatted at 2025-10-23 08:00:11 on dist-test-slave-wkn7"
I20251023 08:00:12.012786 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1761206396810293-18906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1761206396810293-18906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1761206396810293-18906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:12.024523 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:12.025520 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:12.059041 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.190:42281
I20251023 08:00:12.059139 20155 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.190:42281 every 8 connection(s)
I20251023 08:00:12.062606 20156 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:12.072295 20156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139: Bootstrap starting.
I20251023 08:00:12.077220 20156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:12.080988 20156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139: No bootstrap required, opened a new log
I20251023 08:00:12.082825 20156 raft_consensus.cc:359] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9085d66147148448f2c007b78201139" member_type: VOTER }
I20251023 08:00:12.083197 20156 raft_consensus.cc:385] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:12.083374 20156 raft_consensus.cc:740] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9085d66147148448f2c007b78201139, State: Initialized, Role: FOLLOWER
I20251023 08:00:12.083855 20156 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9085d66147148448f2c007b78201139" member_type: VOTER }
I20251023 08:00:12.084260 20156 raft_consensus.cc:399] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 08:00:12.084455 20156 raft_consensus.cc:493] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 08:00:12.084659 20156 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:12.089391 20156 raft_consensus.cc:515] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9085d66147148448f2c007b78201139" member_type: VOTER }
I20251023 08:00:12.089845 20156 leader_election.cc:304] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9085d66147148448f2c007b78201139; no voters: 
I20251023 08:00:12.090871 20156 leader_election.cc:290] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20251023 08:00:12.091177 20159 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:12.092412 20159 raft_consensus.cc:697] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139 [term 1 LEADER]: Becoming Leader. State: Replica: b9085d66147148448f2c007b78201139, State: Running, Role: LEADER
I20251023 08:00:12.093048 20159 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9085d66147148448f2c007b78201139" member_type: VOTER }
I20251023 08:00:12.093729 20156 sys_catalog.cc:565] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 08:00:12.095817 20160 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9085d66147148448f2c007b78201139" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9085d66147148448f2c007b78201139" member_type: VOTER } }
I20251023 08:00:12.095893 20161 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9085d66147148448f2c007b78201139. Latest consensus state: current_term: 1 leader_uuid: "b9085d66147148448f2c007b78201139" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9085d66147148448f2c007b78201139" member_type: VOTER } }
I20251023 08:00:12.096578 20160 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:12.096640 20161 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:12.105455 20165 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 08:00:12.110673 20165 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 08:00:12.111693 18906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 08:00:12.119138 20165 catalog_manager.cc:1357] Generated new cluster ID: b5a048076b594e75bd9ad37d2f8ead46
I20251023 08:00:12.119426 20165 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 08:00:12.136958 20165 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 08:00:12.138202 20165 catalog_manager.cc:1514] Loading token signing keys...
I20251023 08:00:12.151254 20165 catalog_manager.cc:6022] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139: Generated new TSK 0
I20251023 08:00:12.151826 20165 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 08:00:12.178328 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:12.183826 20177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:12.185065 20178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:12.186731 20180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:12.188127 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:12.190121 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:12.190325 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:12.190512 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206412190491 us; error 0 us; skew 500 ppm
I20251023 08:00:12.191031 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:12.193559 18906 webserver.cc:492] Webserver started at http://127.18.118.129:44265/ using document root <none> and password file <none>
I20251023 08:00:12.194022 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:12.194200 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:12.194449 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:12.195657 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1761206396810293-18906-0/minicluster-data/ts-0-root/instance:
uuid: "079261209be44fc8aae3cf0242530b6b"
format_stamp: "Formatted at 2025-10-23 08:00:12 on dist-test-slave-wkn7"
I20251023 08:00:12.200127 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20251023 08:00:12.203627 20185 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:12.204344 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20251023 08:00:12.204617 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1761206396810293-18906-0/minicluster-data/ts-0-root
uuid: "079261209be44fc8aae3cf0242530b6b"
format_stamp: "Formatted at 2025-10-23 08:00:12 on dist-test-slave-wkn7"
I20251023 08:00:12.204898 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1761206396810293-18906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1761206396810293-18906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1761206396810293-18906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:12.226009 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:12.227126 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:12.228492 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:00:12.230813 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:12.231034 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:12.231294 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:12.231463 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:12.270020 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.129:44025
I20251023 08:00:12.270131 20247 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.129:44025 every 8 connection(s)
I20251023 08:00:12.284193 20248 heartbeater.cc:344] Connected to a master server at 127.18.118.190:42281
I20251023 08:00:12.284585 20248 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:12.285288 20248 heartbeater.cc:507] Master 127.18.118.190:42281 requested a full tablet report, sending...
I20251023 08:00:12.287227 20121 ts_manager.cc:194] Registered new tserver with Master: 079261209be44fc8aae3cf0242530b6b (127.18.118.129:44025)
I20251023 08:00:12.287844 18906 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.014713534s
I20251023 08:00:12.289466 20121 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57484
I20251023 08:00:12.307986 18906 tablet_server.cc:178] TabletServer@127.18.118.129:0 shutting down...
I20251023 08:00:12.323112 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:12.338222 18906 tablet_server.cc:195] TabletServer@127.18.118.129:0 shutdown complete.
I20251023 08:00:12.345779 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251023 08:00:12.348353 18906 tablet_server.cc:93] deduplicated master addresses: 127.18.118.190:42281
W20251023 08:00:12.351449 20258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:12.352231 20259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:12.354326 20261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:12.354710 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:12.355521 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:12.355724 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:12.355876 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206412355857 us; error 0 us; skew 500 ppm
I20251023 08:00:12.356357 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:12.358662 18906 webserver.cc:492] Webserver started at http://127.18.118.129:44265/ using document root <none> and password file <none>
I20251023 08:00:12.359120 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:12.359297 18906 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251023 08:00:12.362702 18906 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20251023 08:00:12.365221 20266 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:12.366118 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20251023 08:00:12.366384 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1761206396810293-18906-0/minicluster-data/ts-0-root
uuid: "079261209be44fc8aae3cf0242530b6b"
format_stamp: "Formatted at 2025-10-23 08:00:12 on dist-test-slave-wkn7"
I20251023 08:00:12.366714 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1761206396810293-18906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1761206396810293-18906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1761206396810293-18906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:12.380257 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:12.381240 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:12.382567 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:00:12.384618 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:12.384788 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:12.385043 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:12.385210 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20251023 08:00:12.422705 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.129:44025
I20251023 08:00:12.422780 20328 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.129:44025 every 8 connection(s)
I20251023 08:00:12.428299 18906 tablet_server.cc:178] TabletServer@127.18.118.129:44025 shutting down...
I20251023 08:00:12.443655 20329 heartbeater.cc:344] Connected to a master server at 127.18.118.190:42281
I20251023 08:00:12.444094 20329 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:12.445021 20329 heartbeater.cc:507] Master 127.18.118.190:42281 requested a full tablet report, sending...
I20251023 08:00:12.447871 20121 ts_manager.cc:194] Re-registered known tserver with Master: 079261209be44fc8aae3cf0242530b6b (127.18.118.129:44025)
I20251023 08:00:12.449476 20121 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57504
I20251023 08:00:12.452018 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:12.458467 18906 tablet_server.cc:195] TabletServer@127.18.118.129:44025 shutdown complete.
I20251023 08:00:12.466507 18906 master.cc:561] Master@127.18.118.190:42281 shutting down...
I20251023 08:00:12.483644 18906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139 [term 1 LEADER]: Raft consensus shutting down.
I20251023 08:00:12.484158 18906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:12.484560 18906 tablet_replica.cc:333] T 00000000000000000000000000000000 P b9085d66147148448f2c007b78201139: stopping tablet replica
I20251023 08:00:12.498623 18906 master.cc:583] Master@127.18.118.190:42281 shutdown complete.
[       OK ] TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters (529 ms)
[ RUN      ] TsTabletManagerITest.TestTableStats
I20251023 08:00:12.519650 18906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.18.118.190:38225,127.18.118.189:44091,127.18.118.188:39171
I20251023 08:00:12.520962 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:12.525846 20336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:12.527199 20337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:12.528820 20339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:12.529354 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:12.529958 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:12.530125 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:12.530249 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206412530238 us; error 0 us; skew 500 ppm
I20251023 08:00:12.530709 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:12.532866 18906 webserver.cc:492] Webserver started at http://127.18.118.190:41279/ using document root <none> and password file <none>
I20251023 08:00:12.533370 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:12.533528 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:12.533738 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:12.534726 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/master-0-root/instance:
uuid: "8080b9557fda47c0af876ca344e2477a"
format_stamp: "Formatted at 2025-10-23 08:00:12 on dist-test-slave-wkn7"
I20251023 08:00:12.538987 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20251023 08:00:12.542835 20344 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:12.543764 18906 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20251023 08:00:12.544070 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/master-0-root
uuid: "8080b9557fda47c0af876ca344e2477a"
format_stamp: "Formatted at 2025-10-23 08:00:12 on dist-test-slave-wkn7"
I20251023 08:00:12.544358 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:12.556159 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:12.557206 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:12.592790 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.190:38225
I20251023 08:00:12.592893 20395 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.190:38225 every 8 connection(s)
I20251023 08:00:12.596719 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251023 08:00:12.597329 20396 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20251023 08:00:12.602205 20398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:12.602375 20399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:12.605201 20396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } has no permanent_uuid. Determining permanent_uuid...
W20251023 08:00:12.606058 20401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:12.606101 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:12.607298 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:12.607692 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:12.607898 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206412607880 us; error 0 us; skew 500 ppm
I20251023 08:00:12.608808 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:12.611469 18906 webserver.cc:492] Webserver started at http://127.18.118.189:33889/ using document root <none> and password file <none>
I20251023 08:00:12.611920 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:12.612102 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:12.612349 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:12.613409 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/master-1-root/instance:
uuid: "b303b3b1ff704ead975b9a3f0206f0d1"
format_stamp: "Formatted at 2025-10-23 08:00:12 on dist-test-slave-wkn7"
I20251023 08:00:12.617648 20396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } has no permanent_uuid. Determining permanent_uuid...
I20251023 08:00:12.619295 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
W20251023 08:00:12.619815 20347 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.118.189:44091: connect: Connection refused (error 111)
I20251023 08:00:12.623340 20409 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
W20251023 08:00:12.623296 20396 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.118.189:44091: Network error: Client connection negotiation failed: client connection to 127.18.118.189:44091: connect: Connection refused (error 111)
I20251023 08:00:12.624294 18906 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20251023 08:00:12.624575 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/master-1-root
uuid: "b303b3b1ff704ead975b9a3f0206f0d1"
format_stamp: "Formatted at 2025-10-23 08:00:12 on dist-test-slave-wkn7"
I20251023 08:00:12.624850 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:12.641211 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:12.642189 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:12.662196 20396 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } attempt: 1
W20251023 08:00:12.667347 20396 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.118.189:44091: Network error: Client connection negotiation failed: client connection to 127.18.118.189:44091: connect: Connection refused (error 111)
I20251023 08:00:12.678934 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.189:44091
I20251023 08:00:12.679010 20460 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.189:44091 every 8 connection(s)
I20251023 08:00:12.682556 20461 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:12.682865 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:12.688047 20463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:12.690717 20461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } has no permanent_uuid. Determining permanent_uuid...
W20251023 08:00:12.691749 20464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:12.696224 18906 server_base.cc:1047] running on GCE node
W20251023 08:00:12.696836 20466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:12.697770 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:12.697978 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:12.698140 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206412698119 us; error 0 us; skew 500 ppm
I20251023 08:00:12.698621 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:12.701481 18906 webserver.cc:492] Webserver started at http://127.18.118.188:40499/ using document root <none> and password file <none>
I20251023 08:00:12.701989 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:12.702189 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:12.702515 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:12.703715 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/master-2-root/instance:
uuid: "67329cbbe18040088d9ac6c4ab899d58"
format_stamp: "Formatted at 2025-10-23 08:00:12 on dist-test-slave-wkn7"
I20251023 08:00:12.704196 20461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } has no permanent_uuid. Determining permanent_uuid...
I20251023 08:00:12.707815 20396 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } attempt: 2
I20251023 08:00:12.709040 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20251023 08:00:12.714460 20474 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:12.715782 18906 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20251023 08:00:12.716162 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/master-2-root
uuid: "67329cbbe18040088d9ac6c4ab899d58"
format_stamp: "Formatted at 2025-10-23 08:00:12 on dist-test-slave-wkn7"
I20251023 08:00:12.716799 20461 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } has no permanent_uuid. Determining permanent_uuid...
I20251023 08:00:12.716600 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:12.719730 20396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } has no permanent_uuid. Determining permanent_uuid...
W20251023 08:00:12.724368 20396 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.118.188:39171: Network error: Client connection negotiation failed: client connection to 127.18.118.188:39171: connect: Connection refused (error 111)
W20251023 08:00:12.724917 20461 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.118.188:39171: Network error: Client connection negotiation failed: client connection to 127.18.118.188:39171: connect: Connection refused (error 111)
I20251023 08:00:12.731500 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:12.732525 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:12.758430 20461 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } attempt: 1
I20251023 08:00:12.759768 20396 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } attempt: 1
W20251023 08:00:12.762990 20461 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.118.188:39171: Network error: Client connection negotiation failed: client connection to 127.18.118.188:39171: connect: Connection refused (error 111)
W20251023 08:00:12.764866 20396 consensus_peers.cc:760] Error getting permanent uuid from config peer 127.18.118.188:39171: Network error: Client connection negotiation failed: client connection to 127.18.118.188:39171: connect: Connection refused (error 111)
I20251023 08:00:12.770655 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.188:39171
I20251023 08:00:12.770754 20526 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.188:39171 every 8 connection(s)
I20251023 08:00:12.773612 18906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 08:00:12.774359 20527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:12.779834 20527 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } has no permanent_uuid. Determining permanent_uuid...
I20251023 08:00:12.789726 20527 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } has no permanent_uuid. Determining permanent_uuid...
I20251023 08:00:12.798885 20527 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } has no permanent_uuid. Determining permanent_uuid...
I20251023 08:00:12.803318 20396 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } attempt: 2
I20251023 08:00:12.815944 20527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58: Bootstrap starting.
I20251023 08:00:12.818888 20396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a: Bootstrap starting.
I20251023 08:00:12.820386 20527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:12.823586 20396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:12.824476 20527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58: No bootstrap required, opened a new log
I20251023 08:00:12.826573 20527 raft_consensus.cc:359] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:12.827301 20527 raft_consensus.cc:385] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:12.827554 20527 raft_consensus.cc:740] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67329cbbe18040088d9ac6c4ab899d58, State: Initialized, Role: FOLLOWER
I20251023 08:00:12.827917 20396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a: No bootstrap required, opened a new log
I20251023 08:00:12.828086 20527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:12.829917 20396 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:12.830464 20396 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:12.829990 20534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:12.830717 20396 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8080b9557fda47c0af876ca344e2477a, State: Initialized, Role: FOLLOWER
I20251023 08:00:12.830844 20534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:12.831584 20527 sys_catalog.cc:565] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 08:00:12.831380 20396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:12.833245 20535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:12.834195 20535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:12.834988 20396 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: configured and running, proceeding with master startup.
I20251023 08:00:12.838428 20461 consensus_peers.cc:770] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } attempt: 2
I20251023 08:00:12.852666 18906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
W20251023 08:00:12.858443 20555 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251023 08:00:12.858860 20555 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251023 08:00:12.861594 20461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1: Bootstrap starting.
W20251023 08:00:12.863628 20557 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251023 08:00:12.863976 20557 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251023 08:00:12.866484 20461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:12.870240 20461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1: No bootstrap required, opened a new log
I20251023 08:00:12.872232 20461 raft_consensus.cc:359] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:12.872751 20461 raft_consensus.cc:385] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:12.872939 20461 raft_consensus.cc:740] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b303b3b1ff704ead975b9a3f0206f0d1, State: Initialized, Role: FOLLOWER
I20251023 08:00:12.873490 20461 consensus_queue.cc:260] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:12.875028 20559 sys_catalog.cc:455] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:12.875603 20559 sys_catalog.cc:458] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:12.876214 20461 sys_catalog.cc:565] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 08:00:12.886210 18906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20251023 08:00:12.888038 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:12.890230 20570 catalog_manager.cc:1568] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20251023 08:00:12.890486 20534 raft_consensus.cc:493] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251023 08:00:12.890573 20570 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251023 08:00:12.890983 20534 raft_consensus.cc:515] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:12.892482 20534 leader_election.cc:290] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8080b9557fda47c0af876ca344e2477a (127.18.118.190:38225), b303b3b1ff704ead975b9a3f0206f0d1 (127.18.118.189:44091)
I20251023 08:00:12.893929 20436 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67329cbbe18040088d9ac6c4ab899d58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" is_pre_election: true
I20251023 08:00:12.894176 20371 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67329cbbe18040088d9ac6c4ab899d58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8080b9557fda47c0af876ca344e2477a" is_pre_election: true
I20251023 08:00:12.894682 20436 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67329cbbe18040088d9ac6c4ab899d58 in term 0.
I20251023 08:00:12.895095 20371 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67329cbbe18040088d9ac6c4ab899d58 in term 0.
I20251023 08:00:12.896041 20478 leader_election.cc:304] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67329cbbe18040088d9ac6c4ab899d58, b303b3b1ff704ead975b9a3f0206f0d1; no voters: 
W20251023 08:00:12.896984 20571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:12.897346 20572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:12.897485 20534 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 08:00:12.898182 20534 raft_consensus.cc:493] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 08:00:12.898640 20534 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 0 FOLLOWER]: Advancing to term 1
W20251023 08:00:12.900440 20574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:12.900763 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:12.901857 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:12.902057 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:12.902223 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206412902204 us; error 0 us; skew 500 ppm
I20251023 08:00:12.902717 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:12.905207 18906 webserver.cc:492] Webserver started at http://127.18.118.129:37577/ using document root <none> and password file <none>
I20251023 08:00:12.904879 20534 raft_consensus.cc:515] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:12.905716 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:12.905925 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:12.906177 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:12.906477 20534 leader_election.cc:290] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [CANDIDATE]: Term 1 election: Requested vote from peers 8080b9557fda47c0af876ca344e2477a (127.18.118.190:38225), b303b3b1ff704ead975b9a3f0206f0d1 (127.18.118.189:44091)
I20251023 08:00:12.907244 20371 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67329cbbe18040088d9ac6c4ab899d58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8080b9557fda47c0af876ca344e2477a"
I20251023 08:00:12.907516 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/ts-0-root/instance:
uuid: "4fd4035d6b864683b312d9eba4de5a3e"
format_stamp: "Formatted at 2025-10-23 08:00:12 on dist-test-slave-wkn7"
I20251023 08:00:12.907574 20436 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67329cbbe18040088d9ac6c4ab899d58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b303b3b1ff704ead975b9a3f0206f0d1"
I20251023 08:00:12.907792 20371 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:12.908120 20436 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:12.912599 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.001s
I20251023 08:00:12.914489 20436 raft_consensus.cc:2468] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67329cbbe18040088d9ac6c4ab899d58 in term 1.
I20251023 08:00:12.914489 20371 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67329cbbe18040088d9ac6c4ab899d58 in term 1.
I20251023 08:00:12.915936 20478 leader_election.cc:304] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67329cbbe18040088d9ac6c4ab899d58, b303b3b1ff704ead975b9a3f0206f0d1; no voters: 
I20251023 08:00:12.916221 20579 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:12.916585 20534 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:12.917173 18906 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20251023 08:00:12.917547 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/ts-0-root
uuid: "4fd4035d6b864683b312d9eba4de5a3e"
format_stamp: "Formatted at 2025-10-23 08:00:12 on dist-test-slave-wkn7"
I20251023 08:00:12.917608 20534 raft_consensus.cc:697] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 1 LEADER]: Becoming Leader. State: Replica: 67329cbbe18040088d9ac6c4ab899d58, State: Running, Role: LEADER
I20251023 08:00:12.918007 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:12.918579 20534 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:12.922434 20580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67329cbbe18040088d9ac6c4ab899d58. Latest consensus state: current_term: 1 leader_uuid: "67329cbbe18040088d9ac6c4ab899d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:12.923434 20580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:12.925237 20584 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 08:00:12.930074 20584 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 08:00:12.938120 20436 raft_consensus.cc:1275] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 1 FOLLOWER]: Refusing update from remote peer 67329cbbe18040088d9ac6c4ab899d58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 08:00:12.938403 20371 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 1 FOLLOWER]: Refusing update from remote peer 67329cbbe18040088d9ac6c4ab899d58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 08:00:12.939466 20580 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 08:00:12.940169 20534 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 08:00:12.945740 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:12.947311 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:12.951808 20559 sys_catalog.cc:455] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67329cbbe18040088d9ac6c4ab899d58. Latest consensus state: current_term: 1 leader_uuid: "67329cbbe18040088d9ac6c4ab899d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:12.952819 20559 sys_catalog.cc:458] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:12.954210 20535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67329cbbe18040088d9ac6c4ab899d58. Latest consensus state: current_term: 1 leader_uuid: "67329cbbe18040088d9ac6c4ab899d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:12.954943 20535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:12.955540 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:00:12.958582 20587 mvcc.cc:204] Tried to move back new op lower bound from 7213901467382718464 to 7213901467321188352. Current Snapshot: MvccSnapshot[applied={T|T < 7213901467382718464}]
I20251023 08:00:12.958977 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:12.959206 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:12.959513 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:12.959791 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20251023 08:00:12.965879 20535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "67329cbbe18040088d9ac6c4ab899d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:12.966588 20535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:12.969214 20559 sys_catalog.cc:455] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "67329cbbe18040088d9ac6c4ab899d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:12.969923 20559 sys_catalog.cc:458] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:12.974759 20584 catalog_manager.cc:1357] Generated new cluster ID: e45bbc9edac44b53bbefc3bb992dca7f
I20251023 08:00:12.974453 20580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "67329cbbe18040088d9ac6c4ab899d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:12.975061 20584 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 08:00:12.974577 20534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "67329cbbe18040088d9ac6c4ab899d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:12.975235 20580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:12.975569 20534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:13.001926 20584 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 08:00:13.003420 20584 catalog_manager.cc:1514] Loading token signing keys...
I20251023 08:00:13.025022 20584 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58: Generated new TSK 0
I20251023 08:00:13.025743 20584 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 08:00:13.047598 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.129:34549
I20251023 08:00:13.047703 20654 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.129:34549 every 8 connection(s)
I20251023 08:00:13.057811 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:13.067587 20662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:13.070878 20655 heartbeater.cc:344] Connected to a master server at 127.18.118.188:39171
I20251023 08:00:13.071694 20655 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:13.072957 20655 heartbeater.cc:507] Master 127.18.118.188:39171 requested a full tablet report, sending...
I20251023 08:00:13.076308 20492 ts_manager.cc:194] Registered new tserver with Master: 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549)
I20251023 08:00:13.079581 20492 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53048
I20251023 08:00:13.082094 20657 heartbeater.cc:344] Connected to a master server at 127.18.118.190:38225
I20251023 08:00:13.082772 20657 heartbeater.cc:461] Registering TS with master...
W20251023 08:00:13.084084 20663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:13.084363 20657 heartbeater.cc:507] Master 127.18.118.190:38225 requested a full tablet report, sending...
W20251023 08:00:13.086544 20665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:13.087440 20361 ts_manager.cc:194] Registered new tserver with Master: 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549)
I20251023 08:00:13.087704 20656 heartbeater.cc:344] Connected to a master server at 127.18.118.189:44091
I20251023 08:00:13.088071 20656 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:13.088702 20656 heartbeater.cc:507] Master 127.18.118.189:44091 requested a full tablet report, sending...
I20251023 08:00:13.089253 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:13.090162 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:13.090428 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:13.090664 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206413090644 us; error 0 us; skew 500 ppm
I20251023 08:00:13.091408 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:13.091481 20426 ts_manager.cc:194] Registered new tserver with Master: 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549)
I20251023 08:00:13.094190 18906 webserver.cc:492] Webserver started at http://127.18.118.130:40253/ using document root <none> and password file <none>
I20251023 08:00:13.094673 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:13.094836 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:13.095041 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:13.096063 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/ts-1-root/instance:
uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb"
format_stamp: "Formatted at 2025-10-23 08:00:13 on dist-test-slave-wkn7"
I20251023 08:00:13.100562 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20251023 08:00:13.103897 20670 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:13.104669 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20251023 08:00:13.104967 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/ts-1-root
uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb"
format_stamp: "Formatted at 2025-10-23 08:00:13 on dist-test-slave-wkn7"
I20251023 08:00:13.105290 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:13.121309 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:13.122525 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:13.123704 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:00:13.126338 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:13.126595 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:13.126839 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:13.127081 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20251023 08:00:13.185001 20655 heartbeater.cc:499] Master 127.18.118.188:39171 was elected leader, sending a full tablet report...
I20251023 08:00:13.205289 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.130:38397
I20251023 08:00:13.205386 20732 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.130:38397 every 8 connection(s)
I20251023 08:00:13.219038 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251023 08:00:13.231992 20733 heartbeater.cc:344] Connected to a master server at 127.18.118.188:39171
I20251023 08:00:13.232616 20733 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:13.233855 20733 heartbeater.cc:507] Master 127.18.118.188:39171 requested a full tablet report, sending...
I20251023 08:00:13.236900 20492 ts_manager.cc:194] Registered new tserver with Master: 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397)
I20251023 08:00:13.239099 20492 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53062
I20251023 08:00:13.242595 20734 heartbeater.cc:344] Connected to a master server at 127.18.118.189:44091
I20251023 08:00:13.242982 20734 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:13.243750 20734 heartbeater.cc:507] Master 127.18.118.189:44091 requested a full tablet report, sending...
W20251023 08:00:13.244874 20740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:13.246640 20426 ts_manager.cc:194] Registered new tserver with Master: 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397)
W20251023 08:00:13.247543 20741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:13.248971 20735 heartbeater.cc:344] Connected to a master server at 127.18.118.190:38225
I20251023 08:00:13.249454 20735 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:13.250311 20735 heartbeater.cc:507] Master 127.18.118.190:38225 requested a full tablet report, sending...
I20251023 08:00:13.254652 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:13.255051 20361 ts_manager.cc:194] Registered new tserver with Master: 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397)
W20251023 08:00:13.256033 20743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:13.257128 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:13.257341 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:13.257540 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206413257520 us; error 0 us; skew 500 ppm
I20251023 08:00:13.258018 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:13.260250 18906 webserver.cc:492] Webserver started at http://127.18.118.131:44311/ using document root <none> and password file <none>
I20251023 08:00:13.260707 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:13.260874 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:13.261165 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:13.262215 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/ts-2-root/instance:
uuid: "a5fa082ef7694b6a84e3a4c2add35a2e"
format_stamp: "Formatted at 2025-10-23 08:00:13 on dist-test-slave-wkn7"
I20251023 08:00:13.266350 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20251023 08:00:13.269454 20748 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:13.270155 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20251023 08:00:13.270452 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/ts-2-root
uuid: "a5fa082ef7694b6a84e3a4c2add35a2e"
format_stamp: "Formatted at 2025-10-23 08:00:13 on dist-test-slave-wkn7"
I20251023 08:00:13.270725 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:13.301685 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:13.302878 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:13.304804 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:00:13.307566 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:13.307786 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:13.308012 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:13.308173 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:13.344444 20733 heartbeater.cc:499] Master 127.18.118.188:39171 was elected leader, sending a full tablet report...
I20251023 08:00:13.381752 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.131:44235
I20251023 08:00:13.382015 20810 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.131:44235 every 8 connection(s)
I20251023 08:00:13.406208 20811 heartbeater.cc:344] Connected to a master server at 127.18.118.188:39171
I20251023 08:00:13.406641 20811 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:13.407714 20811 heartbeater.cc:507] Master 127.18.118.188:39171 requested a full tablet report, sending...
I20251023 08:00:13.406584 20812 heartbeater.cc:344] Connected to a master server at 127.18.118.189:44091
I20251023 08:00:13.408715 20812 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:13.409504 20812 heartbeater.cc:507] Master 127.18.118.189:44091 requested a full tablet report, sending...
I20251023 08:00:13.409801 20492 ts_manager.cc:194] Registered new tserver with Master: a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:13.411383 20492 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53064
I20251023 08:00:13.412060 20426 ts_manager.cc:194] Registered new tserver with Master: a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:13.413846 20814 heartbeater.cc:344] Connected to a master server at 127.18.118.190:38225
I20251023 08:00:13.414171 20814 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:13.414669 20814 heartbeater.cc:507] Master 127.18.118.190:38225 requested a full tablet report, sending...
I20251023 08:00:13.416761 20361 ts_manager.cc:194] Registered new tserver with Master: a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:13.416842 18906 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.023612954s
I20251023 08:00:13.465133 20492 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53080:
name: "test-table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  hash_schema {
    columns {
      name: "key"
    }
    num_buckets: 3
    seed: 0
  }
}
W20251023 08:00:13.467154 20492 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 08:00:13.513994 20811 heartbeater.cc:499] Master 127.18.118.188:39171 was elected leader, sending a full tablet report...
I20251023 08:00:13.543502 20776 tablet_service.cc:1505] Processing CreateTablet for tablet 8fc078fb161a4da6b002c30d348ac289 (DEFAULT_TABLE table=test-table [id=ebb365d398e748ffadc1298f5322a54f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20251023 08:00:13.544497 20698 tablet_service.cc:1505] Processing CreateTablet for tablet 8fc078fb161a4da6b002c30d348ac289 (DEFAULT_TABLE table=test-table [id=ebb365d398e748ffadc1298f5322a54f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20251023 08:00:13.545285 20697 tablet_service.cc:1505] Processing CreateTablet for tablet 14613f942a7b438bbf511fe370048d56 (DEFAULT_TABLE table=test-table [id=ebb365d398e748ffadc1298f5322a54f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20251023 08:00:13.546062 20698 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fc078fb161a4da6b002c30d348ac289. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:13.546265 20775 tablet_service.cc:1505] Processing CreateTablet for tablet 14613f942a7b438bbf511fe370048d56 (DEFAULT_TABLE table=test-table [id=ebb365d398e748ffadc1298f5322a54f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20251023 08:00:13.546880 20697 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14613f942a7b438bbf511fe370048d56. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:13.549088 20774 tablet_service.cc:1505] Processing CreateTablet for tablet 0827630b93ca4fb8b649d9567669c1fe (DEFAULT_TABLE table=test-table [id=ebb365d398e748ffadc1298f5322a54f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20251023 08:00:13.550415 20696 tablet_service.cc:1505] Processing CreateTablet for tablet 0827630b93ca4fb8b649d9567669c1fe (DEFAULT_TABLE table=test-table [id=ebb365d398e748ffadc1298f5322a54f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20251023 08:00:13.552541 20696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0827630b93ca4fb8b649d9567669c1fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:13.552572 20776 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fc078fb161a4da6b002c30d348ac289. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:13.554095 20775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14613f942a7b438bbf511fe370048d56. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:13.556020 20774 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0827630b93ca4fb8b649d9567669c1fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:13.567147 20618 tablet_service.cc:1505] Processing CreateTablet for tablet 14613f942a7b438bbf511fe370048d56 (DEFAULT_TABLE table=test-table [id=ebb365d398e748ffadc1298f5322a54f]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20251023 08:00:13.565790 20617 tablet_service.cc:1505] Processing CreateTablet for tablet 0827630b93ca4fb8b649d9567669c1fe (DEFAULT_TABLE table=test-table [id=ebb365d398e748ffadc1298f5322a54f]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20251023 08:00:13.571921 20619 tablet_service.cc:1505] Processing CreateTablet for tablet 8fc078fb161a4da6b002c30d348ac289 (DEFAULT_TABLE table=test-table [id=ebb365d398e748ffadc1298f5322a54f]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20251023 08:00:13.576884 20618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14613f942a7b438bbf511fe370048d56. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:13.579100 20619 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fc078fb161a4da6b002c30d348ac289. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:13.583947 20617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0827630b93ca4fb8b649d9567669c1fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:13.611433 20832 tablet_bootstrap.cc:492] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e: Bootstrap starting.
I20251023 08:00:13.619701 20832 tablet_bootstrap.cc:654] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:13.625943 20833 tablet_bootstrap.cc:492] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e: Bootstrap starting.
I20251023 08:00:13.637123 20834 tablet_bootstrap.cc:492] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Bootstrap starting.
I20251023 08:00:13.642205 20832 tablet_bootstrap.cc:492] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e: No bootstrap required, opened a new log
I20251023 08:00:13.642701 20832 ts_tablet_manager.cc:1403] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e: Time spent bootstrapping tablet: real 0.032s	user 0.017s	sys 0.000s
I20251023 08:00:13.644712 20832 raft_consensus.cc:359] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:13.645275 20832 raft_consensus.cc:385] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:13.645563 20832 raft_consensus.cc:740] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5fa082ef7694b6a84e3a4c2add35a2e, State: Initialized, Role: FOLLOWER
I20251023 08:00:13.646476 20833 tablet_bootstrap.cc:654] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:13.646324 20832 consensus_queue.cc:260] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:13.652019 20832 ts_tablet_manager.cc:1434] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e: Time spent starting tablet: real 0.009s	user 0.000s	sys 0.006s
I20251023 08:00:13.653301 20832 tablet_bootstrap.cc:492] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e: Bootstrap starting.
I20251023 08:00:13.658938 20832 tablet_bootstrap.cc:654] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:13.665493 20833 tablet_bootstrap.cc:492] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e: No bootstrap required, opened a new log
I20251023 08:00:13.665894 20834 tablet_bootstrap.cc:654] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:13.665987 20833 ts_tablet_manager.cc:1403] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e: Time spent bootstrapping tablet: real 0.041s	user 0.011s	sys 0.006s
I20251023 08:00:13.668809 20836 raft_consensus.cc:493] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 08:00:13.668917 20833 raft_consensus.cc:359] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:13.670217 20833 raft_consensus.cc:385] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:13.670624 20833 raft_consensus.cc:740] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fd4035d6b864683b312d9eba4de5a3e, State: Initialized, Role: FOLLOWER
I20251023 08:00:13.669885 20836 raft_consensus.cc:515] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:13.675946 20836 leader_election.cc:290] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549), 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397)
I20251023 08:00:13.677089 20832 tablet_bootstrap.cc:492] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e: No bootstrap required, opened a new log
I20251023 08:00:13.677837 20832 ts_tablet_manager.cc:1403] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e: Time spent bootstrapping tablet: real 0.025s	user 0.020s	sys 0.003s
I20251023 08:00:13.680233 20834 tablet_bootstrap.cc:492] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: No bootstrap required, opened a new log
I20251023 08:00:13.680785 20832 raft_consensus.cc:359] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:13.680784 20834 ts_tablet_manager.cc:1403] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Time spent bootstrapping tablet: real 0.046s	user 0.008s	sys 0.006s
I20251023 08:00:13.671401 20833 consensus_queue.cc:260] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:13.685132 20834 raft_consensus.cc:359] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:13.685971 20834 raft_consensus.cc:385] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:13.686288 20834 raft_consensus.cc:740] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, State: Initialized, Role: FOLLOWER
I20251023 08:00:13.686964 20832 raft_consensus.cc:385] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:13.687362 20832 raft_consensus.cc:740] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5fa082ef7694b6a84e3a4c2add35a2e, State: Initialized, Role: FOLLOWER
I20251023 08:00:13.688055 20832 consensus_queue.cc:260] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:13.690344 20832 ts_tablet_manager.cc:1434] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.004s
I20251023 08:00:13.691475 20832 tablet_bootstrap.cc:492] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e: Bootstrap starting.
I20251023 08:00:13.691653 20708 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" is_pre_election: true
I20251023 08:00:13.694257 20630 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
W20251023 08:00:13.696069 20751 leader_election.cc:343] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549): Illegal state: must be running to vote when last-logged opid is not known
I20251023 08:00:13.697337 20834 consensus_queue.cc:260] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:13.698343 20708 raft_consensus.cc:2468] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5fa082ef7694b6a84e3a4c2add35a2e in term 0.
I20251023 08:00:13.702816 20752 leader_election.cc:304] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 4fd4035d6b864683b312d9eba4de5a3e
I20251023 08:00:13.703262 20832 tablet_bootstrap.cc:654] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:13.704360 20836 raft_consensus.cc:2804] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 08:00:13.704854 20836 raft_consensus.cc:493] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 08:00:13.705325 20836 raft_consensus.cc:3060] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:13.711122 20834 ts_tablet_manager.cc:1434] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Time spent starting tablet: real 0.029s	user 0.012s	sys 0.000s
I20251023 08:00:13.713305 20836 raft_consensus.cc:515] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:13.717617 20630 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e"
I20251023 08:00:13.719822 20708 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb"
I20251023 08:00:13.720651 20708 raft_consensus.cc:3060] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:13.724025 20834 tablet_bootstrap.cc:492] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Bootstrap starting.
W20251023 08:00:13.726039 20751 leader_election.cc:343] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549): Illegal state: must be running to vote when last-logged opid is not known
I20251023 08:00:13.731585 20708 raft_consensus.cc:2468] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5fa082ef7694b6a84e3a4c2add35a2e in term 1.
I20251023 08:00:13.733520 20834 tablet_bootstrap.cc:654] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:13.738348 20833 ts_tablet_manager.cc:1434] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e: Time spent starting tablet: real 0.072s	user 0.010s	sys 0.009s
I20251023 08:00:13.739437 20833 tablet_bootstrap.cc:492] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e: Bootstrap starting.
I20251023 08:00:13.732231 20836 leader_election.cc:290] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 1 election: Requested vote from peers 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549), 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397)
I20251023 08:00:13.741336 20752 leader_election.cc:304] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 4fd4035d6b864683b312d9eba4de5a3e
I20251023 08:00:13.742501 20836 raft_consensus.cc:2804] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:13.748627 20833 tablet_bootstrap.cc:654] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:13.757409 20832 tablet_bootstrap.cc:492] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e: No bootstrap required, opened a new log
I20251023 08:00:13.757759 20833 tablet_bootstrap.cc:492] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e: No bootstrap required, opened a new log
I20251023 08:00:13.758080 20832 ts_tablet_manager.cc:1403] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e: Time spent bootstrapping tablet: real 0.067s	user 0.012s	sys 0.018s
I20251023 08:00:13.758306 20833 ts_tablet_manager.cc:1403] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.003s
I20251023 08:00:13.760108 20840 raft_consensus.cc:493] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 08:00:13.760783 20840 raft_consensus.cc:515] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:13.761242 20836 raft_consensus.cc:697] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 1 LEADER]: Becoming Leader. State: Replica: a5fa082ef7694b6a84e3a4c2add35a2e, State: Running, Role: LEADER
I20251023 08:00:13.762389 20836 consensus_queue.cc:237] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:13.763593 20834 tablet_bootstrap.cc:492] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: No bootstrap required, opened a new log
I20251023 08:00:13.764117 20834 ts_tablet_manager.cc:1403] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Time spent bootstrapping tablet: real 0.040s	user 0.009s	sys 0.009s
I20251023 08:00:13.768640 20834 raft_consensus.cc:359] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:13.769876 20834 raft_consensus.cc:385] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:13.770337 20834 raft_consensus.cc:740] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, State: Initialized, Role: FOLLOWER
I20251023 08:00:13.762034 20833 raft_consensus.cc:359] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:13.772082 20833 raft_consensus.cc:385] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:13.774397 20833 raft_consensus.cc:740] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fd4035d6b864683b312d9eba4de5a3e, State: Initialized, Role: FOLLOWER
I20251023 08:00:13.774765 20832 raft_consensus.cc:359] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:13.775843 20832 raft_consensus.cc:385] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:13.775512 20833 consensus_queue.cc:260] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:13.773449 20844 raft_consensus.cc:493] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 08:00:13.769871 20840 leader_election.cc:290] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397), a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:13.777812 20844 raft_consensus.cc:515] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:13.776289 20832 raft_consensus.cc:740] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5fa082ef7694b6a84e3a4c2add35a2e, State: Initialized, Role: FOLLOWER
I20251023 08:00:13.772854 20834 consensus_queue.cc:260] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:13.784580 20834 ts_tablet_manager.cc:1434] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.008s
I20251023 08:00:13.785913 20834 tablet_bootstrap.cc:492] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Bootstrap starting.
I20251023 08:00:13.803754 20834 tablet_bootstrap.cc:654] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:13.803601 20832 consensus_queue.cc:260] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:13.807165 20708 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" is_pre_election: true
I20251023 08:00:13.807878 20708 raft_consensus.cc:2468] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fd4035d6b864683b312d9eba4de5a3e in term 0.
I20251023 08:00:13.808018 20844 leader_election.cc:290] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397), 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549)
I20251023 08:00:13.809921 20630 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
I20251023 08:00:13.812484 20708 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" is_pre_election: true
W20251023 08:00:13.813582 20751 leader_election.cc:343] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549): Illegal state: must be running to vote when last-logged opid is not known
W20251023 08:00:13.814363 20752 leader_election.cc:343] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397): Illegal state: must be running to vote when last-logged opid is not known
I20251023 08:00:13.814859 20752 leader_election.cc:304] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 4fd4035d6b864683b312d9eba4de5a3e, 7fdcd2a5f83a48cb8f77f5f4fd8929bb
I20251023 08:00:13.814890 20586 leader_election.cc:304] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fd4035d6b864683b312d9eba4de5a3e, 7fdcd2a5f83a48cb8f77f5f4fd8929bb; no voters: 
I20251023 08:00:13.816464 20834 tablet_bootstrap.cc:492] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: No bootstrap required, opened a new log
I20251023 08:00:13.816828 20840 raft_consensus.cc:2804] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 08:00:13.814859 20832 ts_tablet_manager.cc:1434] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.002s
I20251023 08:00:13.817543 20840 raft_consensus.cc:493] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 08:00:13.818020 20840 raft_consensus.cc:3060] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:13.824389 20491 catalog_manager.cc:5649] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e reported cstate change: term changed from 0 to 1, leader changed from <none> to a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131). New cstate: current_term: 1 leader_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } health_report { overall_health: HEALTHY } } }
I20251023 08:00:13.824748 20840 raft_consensus.cc:515] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:13.827880 20708 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb"
I20251023 08:00:13.828446 20708 raft_consensus.cc:3060] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:13.825393 20834 ts_tablet_manager.cc:1403] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.008s
I20251023 08:00:13.833659 20786 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" is_pre_election: true
I20251023 08:00:13.834460 20786 raft_consensus.cc:2468] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fd4035d6b864683b312d9eba4de5a3e in term 0.
I20251023 08:00:13.835132 20833 ts_tablet_manager.cc:1434] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e: Time spent starting tablet: real 0.076s	user 0.015s	sys 0.006s
I20251023 08:00:13.835609 20834 raft_consensus.cc:359] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:13.836609 20834 raft_consensus.cc:385] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:13.836823 20834 raft_consensus.cc:740] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, State: Initialized, Role: FOLLOWER
I20251023 08:00:13.816877 20844 raft_consensus.cc:2749] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251023 08:00:13.839651 20708 raft_consensus.cc:2468] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fd4035d6b864683b312d9eba4de5a3e in term 1.
I20251023 08:00:13.840698 20586 leader_election.cc:304] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fd4035d6b864683b312d9eba4de5a3e, 7fdcd2a5f83a48cb8f77f5f4fd8929bb; no voters: 
I20251023 08:00:13.841892 20840 leader_election.cc:290] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 1 election: Requested vote from peers 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397), a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:13.842324 20833 tablet_bootstrap.cc:492] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e: Bootstrap starting.
I20251023 08:00:13.842391 20840 raft_consensus.cc:2804] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:13.842957 20840 raft_consensus.cc:697] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 1 LEADER]: Becoming Leader. State: Replica: 4fd4035d6b864683b312d9eba4de5a3e, State: Running, Role: LEADER
I20251023 08:00:13.843739 20840 consensus_queue.cc:237] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:13.847645 20834 consensus_queue.cc:260] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:13.850481 20834 ts_tablet_manager.cc:1434] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20251023 08:00:13.857084 20630 raft_consensus.cc:3060] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:13.866125 20833 tablet_bootstrap.cc:654] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:13.866269 20844 consensus_queue.cc:1048] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 08:00:13.883916 20840 raft_consensus.cc:493] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 08:00:13.884471 20840 raft_consensus.cc:515] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:13.887259 20840 leader_election.cc:290] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397), a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:13.888538 20833 tablet_bootstrap.cc:492] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e: No bootstrap required, opened a new log
I20251023 08:00:13.889052 20833 ts_tablet_manager.cc:1403] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e: Time spent bootstrapping tablet: real 0.047s	user 0.010s	sys 0.002s
I20251023 08:00:13.892143 20833 raft_consensus.cc:359] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:13.892871 20833 raft_consensus.cc:385] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:13.893283 20833 raft_consensus.cc:740] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fd4035d6b864683b312d9eba4de5a3e, State: Initialized, Role: FOLLOWER
I20251023 08:00:13.887167 20708 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" is_pre_election: true
I20251023 08:00:13.896310 20570 catalog_manager.cc:1269] Loaded cluster ID: e45bbc9edac44b53bbefc3bb992dca7f
I20251023 08:00:13.896641 20570 catalog_manager.cc:1562] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1: loading cluster ID for follower catalog manager: success
I20251023 08:00:13.896535 20836 consensus_queue.cc:1048] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 08:00:13.897262 20833 consensus_queue.cc:260] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:13.899178 20786 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e"
I20251023 08:00:13.899771 20786 raft_consensus.cc:3060] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:13.900226 20555 catalog_manager.cc:1269] Loaded cluster ID: e45bbc9edac44b53bbefc3bb992dca7f
I20251023 08:00:13.900683 20785 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" is_pre_election: true
I20251023 08:00:13.903836 20555 catalog_manager.cc:1562] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a: loading cluster ID for follower catalog manager: success
I20251023 08:00:13.907769 20786 raft_consensus.cc:2468] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fd4035d6b864683b312d9eba4de5a3e in term 1.
I20251023 08:00:13.899845 20833 ts_tablet_manager.cc:1434] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20251023 08:00:13.904970 20581 leader_election.cc:304] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4fd4035d6b864683b312d9eba4de5a3e; no voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, a5fa082ef7694b6a84e3a4c2add35a2e
I20251023 08:00:13.919520 20555 catalog_manager.cc:1584] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a: acquiring CA information for follower catalog manager: success
I20251023 08:00:13.926111 20555 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251023 08:00:13.926919 20570 catalog_manager.cc:1584] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1: acquiring CA information for follower catalog manager: success
I20251023 08:00:13.908684 20490 catalog_manager.cc:5649] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129). New cstate: current_term: 1 leader_uuid: "4fd4035d6b864683b312d9eba4de5a3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } health_report { overall_health: UNKNOWN } } }
I20251023 08:00:13.933414 20853 raft_consensus.cc:2749] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251023 08:00:13.934825 20841 raft_consensus.cc:493] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 08:00:13.935369 20841 raft_consensus.cc:515] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:13.942728 20841 leader_election.cc:290] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235), 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549)
I20251023 08:00:13.969673 20630 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
I20251023 08:00:13.970247 20630 raft_consensus.cc:2468] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in term 0.
I20251023 08:00:13.971330 20673 leader_election.cc:304] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fd4035d6b864683b312d9eba4de5a3e, 7fdcd2a5f83a48cb8f77f5f4fd8929bb; no voters: 
I20251023 08:00:13.972086 20841 raft_consensus.cc:2804] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 08:00:13.972415 20841 raft_consensus.cc:493] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 08:00:13.972678 20841 raft_consensus.cc:3060] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:13.978081 20853 consensus_queue.cc:1048] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 08:00:13.980362 20841 raft_consensus.cc:515] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:13.981271 20853 consensus_queue.cc:1048] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 08:00:13.982654 20841 leader_election.cc:290] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 1 election: Requested vote from peers a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235), 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549)
I20251023 08:00:13.986543 20570 catalog_manager.cc:1612] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251023 08:00:13.988843 20630 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e"
I20251023 08:00:13.989631 20630 raft_consensus.cc:3060] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:13.991508 20785 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" is_pre_election: true
I20251023 08:00:13.992343 20785 raft_consensus.cc:2468] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in term 0.
I20251023 08:00:13.993597 20784 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e"
I20251023 08:00:13.994160 20784 raft_consensus.cc:3060] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:14.005980 20784 raft_consensus.cc:2468] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in term 1.
I20251023 08:00:14.014966 20671 leader_election.cc:304] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 
I20251023 08:00:14.015851 20841 raft_consensus.cc:2804] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:14.015091 20630 raft_consensus.cc:2468] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in term 1.
I20251023 08:00:14.032641 20841 raft_consensus.cc:697] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 1 LEADER]: Becoming Leader. State: Replica: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, State: Running, Role: LEADER
I20251023 08:00:14.033635 20841 consensus_queue.cc:237] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:14.043985 20490 catalog_manager.cc:5649] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130). New cstate: current_term: 1 leader_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } health_report { overall_health: UNKNOWN } } }
I20251023 08:00:14.140286 20841 consensus_queue.cc:1048] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 08:00:14.157269 20841 consensus_queue.cc:1048] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 08:00:14.738566 20630 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe"
dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e"
 from {username='slave'} at 127.0.0.1:35602
I20251023 08:00:14.739156 20630 raft_consensus.cc:591] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 1 LEADER]: Received request to step down
I20251023 08:00:14.739418 20630 raft_consensus.cc:3055] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 1 LEADER]: Stepping down as leader of term 1
I20251023 08:00:14.739627 20630 raft_consensus.cc:740] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4fd4035d6b864683b312d9eba4de5a3e, State: Running, Role: LEADER
I20251023 08:00:14.740217 20630 consensus_queue.cc:260] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:14.741169 20630 raft_consensus.cc:3060] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:14.887257 20844 raft_consensus.cc:493] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4fd4035d6b864683b312d9eba4de5a3e)
I20251023 08:00:14.887741 20844 raft_consensus.cc:515] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:14.889250 20844 leader_election.cc:290] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549), 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397)
I20251023 08:00:14.890118 20630 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
I20251023 08:00:14.890417 20865 raft_consensus.cc:493] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4fd4035d6b864683b312d9eba4de5a3e)
I20251023 08:00:14.890461 20707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" is_pre_election: true
I20251023 08:00:14.890877 20630 raft_consensus.cc:2468] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5fa082ef7694b6a84e3a4c2add35a2e in term 2.
I20251023 08:00:14.891067 20865 raft_consensus.cc:515] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:14.892096 20707 raft_consensus.cc:2468] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5fa082ef7694b6a84e3a4c2add35a2e in term 1.
I20251023 08:00:14.892170 20751 leader_election.cc:304] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fd4035d6b864683b312d9eba4de5a3e, a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 
I20251023 08:00:14.893170 20844 raft_consensus.cc:2804] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 08:00:14.893539 20844 raft_consensus.cc:493] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4fd4035d6b864683b312d9eba4de5a3e)
I20251023 08:00:14.893868 20865 leader_election.cc:290] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549), a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:14.893903 20844 raft_consensus.cc:3060] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:14.895126 20786 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" is_pre_election: true
I20251023 08:00:14.894266 20630 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
I20251023 08:00:14.895867 20630 raft_consensus.cc:2468] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in term 2.
I20251023 08:00:14.896893 20673 leader_election.cc:304] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fd4035d6b864683b312d9eba4de5a3e, 7fdcd2a5f83a48cb8f77f5f4fd8929bb; no voters: 
I20251023 08:00:14.897950 20865 raft_consensus.cc:2804] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 08:00:14.898300 20865 raft_consensus.cc:493] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4fd4035d6b864683b312d9eba4de5a3e)
I20251023 08:00:14.898672 20865 raft_consensus.cc:3060] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:14.903616 20844 raft_consensus.cc:515] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:14.904881 20786 raft_consensus.cc:2393] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in current term 2: Already voted for candidate a5fa082ef7694b6a84e3a4c2add35a2e in this term.
I20251023 08:00:14.905928 20844 leader_election.cc:290] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 2 election: Requested vote from peers 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549), 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397)
I20251023 08:00:14.906302 20630 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e"
I20251023 08:00:14.907095 20707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb"
I20251023 08:00:14.913688 20865 raft_consensus.cc:515] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:14.914904 20707 raft_consensus.cc:2393] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a5fa082ef7694b6a84e3a4c2add35a2e in current term 2: Already voted for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in this term.
I20251023 08:00:14.914826 20630 raft_consensus.cc:2468] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5fa082ef7694b6a84e3a4c2add35a2e in term 2.
I20251023 08:00:14.915969 20751 leader_election.cc:304] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fd4035d6b864683b312d9eba4de5a3e, a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 
I20251023 08:00:14.916417 20865 leader_election.cc:290] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 2 election: Requested vote from peers 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549), a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:14.916929 20630 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e"
I20251023 08:00:14.917038 20844 raft_consensus.cc:2804] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Leader election won for term 2
I20251023 08:00:14.917780 20630 raft_consensus.cc:2393] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in current term 2: Already voted for candidate a5fa082ef7694b6a84e3a4c2add35a2e in this term.
I20251023 08:00:14.917761 20786 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e"
I20251023 08:00:14.918423 20844 raft_consensus.cc:697] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 LEADER]: Becoming Leader. State: Replica: a5fa082ef7694b6a84e3a4c2add35a2e, State: Running, Role: LEADER
I20251023 08:00:14.919853 20844 consensus_queue.cc:237] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:14.927891 20671 leader_election.cc:304] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb; no voters: 4fd4035d6b864683b312d9eba4de5a3e, a5fa082ef7694b6a84e3a4c2add35a2e
I20251023 08:00:14.928818 20865 raft_consensus.cc:2749] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20251023 08:00:14.929930 20491 catalog_manager.cc:5649] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e reported cstate change: term changed from 1 to 2, leader changed from 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129) to a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131). New cstate: current_term: 2 leader_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } health_report { overall_health: HEALTHY } } }
I20251023 08:00:14.966852 20707 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289"
dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb"
 from {username='slave'} at 127.0.0.1:43726
I20251023 08:00:14.967517 20707 raft_consensus.cc:591] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 1 LEADER]: Received request to step down
I20251023 08:00:14.967845 20707 raft_consensus.cc:3055] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 1 LEADER]: Stepping down as leader of term 1
I20251023 08:00:14.968133 20707 raft_consensus.cc:740] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, State: Running, Role: LEADER
I20251023 08:00:14.968878 20707 consensus_queue.cc:260] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:14.969897 20707 raft_consensus.cc:3060] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:15.006238 20630 raft_consensus.cc:1275] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 2 FOLLOWER]: Refusing update from remote peer a5fa082ef7694b6a84e3a4c2add35a2e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 08:00:15.007539 20844 consensus_queue.cc:1048] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 08:00:15.017885 20707 raft_consensus.cc:1275] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Refusing update from remote peer a5fa082ef7694b6a84e3a4c2add35a2e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 08:00:15.021209 20844 consensus_queue.cc:1048] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 08:00:15.122942 20853 raft_consensus.cc:493] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7fdcd2a5f83a48cb8f77f5f4fd8929bb)
I20251023 08:00:15.123418 20853 raft_consensus.cc:515] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:15.125017 20853 leader_election.cc:290] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235), 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397)
I20251023 08:00:15.125802 20786 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" is_pre_election: true
I20251023 08:00:15.125984 20707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" is_pre_election: true
I20251023 08:00:15.126569 20707 raft_consensus.cc:2468] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fd4035d6b864683b312d9eba4de5a3e in term 2.
I20251023 08:00:15.127506 20586 leader_election.cc:304] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4fd4035d6b864683b312d9eba4de5a3e, 7fdcd2a5f83a48cb8f77f5f4fd8929bb; no voters: a5fa082ef7694b6a84e3a4c2add35a2e
I20251023 08:00:15.127677 20844 raft_consensus.cc:493] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7fdcd2a5f83a48cb8f77f5f4fd8929bb)
I20251023 08:00:15.128154 20853 raft_consensus.cc:2804] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 08:00:15.128154 20844 raft_consensus.cc:515] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:15.128718 20853 raft_consensus.cc:493] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7fdcd2a5f83a48cb8f77f5f4fd8929bb)
I20251023 08:00:15.129159 20853 raft_consensus.cc:3060] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:15.130345 20844 leader_election.cc:290] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397), 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549)
I20251023 08:00:15.131115 20707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" is_pre_election: true
I20251023 08:00:15.131704 20707 raft_consensus.cc:2468] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5fa082ef7694b6a84e3a4c2add35a2e in term 2.
I20251023 08:00:15.131835 20630 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
I20251023 08:00:15.132866 20752 leader_election.cc:304] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 
I20251023 08:00:15.133749 20844 raft_consensus.cc:2804] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 08:00:15.134127 20844 raft_consensus.cc:493] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7fdcd2a5f83a48cb8f77f5f4fd8929bb)
I20251023 08:00:15.134445 20844 raft_consensus.cc:3060] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:15.135414 20853 raft_consensus.cc:515] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:15.137696 20630 raft_consensus.cc:2393] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5fa082ef7694b6a84e3a4c2add35a2e in current term 2: Already voted for candidate 4fd4035d6b864683b312d9eba4de5a3e in this term.
I20251023 08:00:15.137908 20786 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e"
I20251023 08:00:15.140000 20853 leader_election.cc:290] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 2 election: Requested vote from peers a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235), 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397)
I20251023 08:00:15.141434 20707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb"
I20251023 08:00:15.145290 20844 raft_consensus.cc:515] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:15.146561 20786 raft_consensus.cc:2393] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4fd4035d6b864683b312d9eba4de5a3e in current term 2: Already voted for candidate a5fa082ef7694b6a84e3a4c2add35a2e in this term.
I20251023 08:00:15.148820 20844 leader_election.cc:290] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 2 election: Requested vote from peers 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397), 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549)
I20251023 08:00:15.149550 20708 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb"
I20251023 08:00:15.149966 20707 raft_consensus.cc:2468] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fd4035d6b864683b312d9eba4de5a3e in term 2.
I20251023 08:00:15.150372 20630 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e"
I20251023 08:00:15.151125 20630 raft_consensus.cc:2393] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a5fa082ef7694b6a84e3a4c2add35a2e in current term 2: Already voted for candidate 4fd4035d6b864683b312d9eba4de5a3e in this term.
I20251023 08:00:15.151070 20586 leader_election.cc:304] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4fd4035d6b864683b312d9eba4de5a3e, 7fdcd2a5f83a48cb8f77f5f4fd8929bb; no voters: a5fa082ef7694b6a84e3a4c2add35a2e
I20251023 08:00:15.151981 20853 raft_consensus.cc:2804] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 2 FOLLOWER]: Leader election won for term 2
I20251023 08:00:15.152146 20751 leader_election.cc:304] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 4fd4035d6b864683b312d9eba4de5a3e, 7fdcd2a5f83a48cb8f77f5f4fd8929bb
I20251023 08:00:15.152498 20853 raft_consensus.cc:697] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 2 LEADER]: Becoming Leader. State: Replica: 4fd4035d6b864683b312d9eba4de5a3e, State: Running, Role: LEADER
I20251023 08:00:15.153079 20844 raft_consensus.cc:2749] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20251023 08:00:15.153419 20853 consensus_queue.cc:237] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:15.160513 20492 catalog_manager.cc:5649] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e reported cstate change: term changed from 1 to 2, leader changed from 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130) to 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129). New cstate: current_term: 2 leader_uuid: "4fd4035d6b864683b312d9eba4de5a3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } health_report { overall_health: HEALTHY } } }
I20251023 08:00:15.205736 20786 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe"
dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e"
 from {username='slave'} at 127.0.0.1:50884
I20251023 08:00:15.206398 20786 raft_consensus.cc:591] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 LEADER]: Received request to step down
I20251023 08:00:15.206669 20786 raft_consensus.cc:3055] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 LEADER]: Stepping down as leader of term 2
I20251023 08:00:15.206866 20786 raft_consensus.cc:740] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a5fa082ef7694b6a84e3a4c2add35a2e, State: Running, Role: LEADER
I20251023 08:00:15.207547 20786 consensus_queue.cc:260] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:15.208647 20786 raft_consensus.cc:3060] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Advancing to term 3
I20251023 08:00:15.255590 20786 raft_consensus.cc:1275] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Refusing update from remote peer 4fd4035d6b864683b312d9eba4de5a3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 08:00:15.256860 20853 consensus_queue.cc:1048] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 08:00:15.265161 20707 raft_consensus.cc:1275] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Refusing update from remote peer 4fd4035d6b864683b312d9eba4de5a3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 08:00:15.267691 20900 consensus_queue.cc:1048] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 08:00:15.305801 20902 raft_consensus.cc:493] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a5fa082ef7694b6a84e3a4c2add35a2e)
I20251023 08:00:15.306762 20902 raft_consensus.cc:515] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:15.309893 20630 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
I20251023 08:00:15.311403 20902 leader_election.cc:290] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549), a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:15.312088 20786 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" is_pre_election: true
I20251023 08:00:15.312636 20786 raft_consensus.cc:2468] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in term 3.
I20251023 08:00:15.313555 20671 leader_election.cc:304] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 4fd4035d6b864683b312d9eba4de5a3e
I20251023 08:00:15.314311 20902 raft_consensus.cc:2804] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251023 08:00:15.314682 20902 raft_consensus.cc:493] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Starting leader election (detected failure of leader a5fa082ef7694b6a84e3a4c2add35a2e)
I20251023 08:00:15.315044 20902 raft_consensus.cc:3060] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Advancing to term 3
I20251023 08:00:15.321177 20902 raft_consensus.cc:515] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:15.323529 20902 leader_election.cc:290] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 3 election: Requested vote from peers 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549), a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:15.323940 20630 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e"
I20251023 08:00:15.324064 20786 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e"
I20251023 08:00:15.342871 20786 raft_consensus.cc:2468] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in term 3.
I20251023 08:00:15.344158 20671 leader_election.cc:304] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 4fd4035d6b864683b312d9eba4de5a3e
I20251023 08:00:15.347051 20902 raft_consensus.cc:2804] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Leader election won for term 3
I20251023 08:00:15.348605 20902 raft_consensus.cc:697] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 LEADER]: Becoming Leader. State: Replica: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, State: Running, Role: LEADER
I20251023 08:00:15.349560 20902 consensus_queue.cc:237] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:15.459049 20903 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 67329cbbe18040088d9ac6c4ab899d58)
I20251023 08:00:15.459657 20903 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:15.461629 20903 leader_election.cc:290] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b303b3b1ff704ead975b9a3f0206f0d1 (127.18.118.189:44091), 67329cbbe18040088d9ac6c4ab899d58 (127.18.118.188:39171)
I20251023 08:00:15.471777 20436 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8080b9557fda47c0af876ca344e2477a" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: false dest_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" is_pre_election: true
I20251023 08:00:15.480988 20902 raft_consensus.cc:493] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4fd4035d6b864683b312d9eba4de5a3e)
I20251023 08:00:15.481686 20902 raft_consensus.cc:515] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:15.495450 20904 raft_consensus.cc:493] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 67329cbbe18040088d9ac6c4ab899d58)
I20251023 08:00:15.496384 20904 raft_consensus.cc:515] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:15.500244 20370 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: false dest_uuid: "8080b9557fda47c0af876ca344e2477a" is_pre_election: true
I20251023 08:00:15.501124 20370 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b303b3b1ff704ead975b9a3f0206f0d1 in term 1.
I20251023 08:00:15.502806 20412 leader_election.cc:304] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8080b9557fda47c0af876ca344e2477a, b303b3b1ff704ead975b9a3f0206f0d1; no voters: 
I20251023 08:00:15.518959 20906 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 08:00:15.519582 20906 raft_consensus.cc:493] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 67329cbbe18040088d9ac6c4ab899d58)
I20251023 08:00:15.520108 20906 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:15.526274 20904 leader_election.cc:290] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8080b9557fda47c0af876ca344e2477a (127.18.118.190:38225), 67329cbbe18040088d9ac6c4ab899d58 (127.18.118.188:39171)
I20251023 08:00:15.540761 20906 raft_consensus.cc:515] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:15.544646 20370 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: false dest_uuid: "8080b9557fda47c0af876ca344e2477a"
I20251023 08:00:15.545501 20370 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:15.555190 20370 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b303b3b1ff704ead975b9a3f0206f0d1 in term 2.
I20251023 08:00:15.556761 20412 leader_election.cc:304] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8080b9557fda47c0af876ca344e2477a, b303b3b1ff704ead975b9a3f0206f0d1; no voters: 
I20251023 08:00:15.558018 20904 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 08:00:15.567862 20904 raft_consensus.cc:697] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 2 LEADER]: Becoming Leader. State: Replica: b303b3b1ff704ead975b9a3f0206f0d1, State: Running, Role: LEADER
I20251023 08:00:15.569015 20904 consensus_queue.cc:237] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:15.605918 20902 leader_election.cc:290] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235), 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549)
I20251023 08:00:15.607439 20902 raft_consensus.cc:493] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a5fa082ef7694b6a84e3a4c2add35a2e)
I20251023 08:00:15.608067 20902 raft_consensus.cc:515] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:15.626003 20906 leader_election.cc:290] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [CANDIDATE]: Term 2 election: Requested vote from peers 8080b9557fda47c0af876ca344e2477a (127.18.118.190:38225), 67329cbbe18040088d9ac6c4ab899d58 (127.18.118.188:39171)
I20251023 08:00:15.679814 20370 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 2 FOLLOWER]: Refusing update from remote peer b303b3b1ff704ead975b9a3f0206f0d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 2 index: 12. (index mismatch)
I20251023 08:00:15.681782 20904 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20251023 08:00:15.688932 20903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b303b3b1ff704ead975b9a3f0206f0d1. Latest consensus state: current_term: 2 leader_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:15.689906 20903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:15.803241 20904 sys_catalog.cc:455] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:15.804270 20904 sys_catalog.cc:458] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: This master's current role is: LEADER
W20251023 08:00:16.002203 20410 leader_election.cc:336] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 67329cbbe18040088d9ac6c4ab899d58 (127.18.118.188:39171): Timed out: RequestConsensusVote RPC to 127.18.118.188:39171 timed out after 0.221s (SENT)
W20251023 08:00:16.058107 20345 leader_election.cc:336] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 67329cbbe18040088d9ac6c4ab899d58 (127.18.118.188:39171): Timed out: RequestConsensusVote RPC to 127.18.118.188:39171 timed out after 0.277s (SENT)
I20251023 08:00:16.058804 20345 leader_election.cc:304] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8080b9557fda47c0af876ca344e2477a; no voters: 67329cbbe18040088d9ac6c4ab899d58, b303b3b1ff704ead975b9a3f0206f0d1
I20251023 08:00:16.060325 20903 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20251023 08:00:16.121361 20673 leader_election.cc:336] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549): Timed out: RequestConsensusVote RPC to 127.18.118.129:34549 timed out after 0.240s (SENT)
W20251023 08:00:16.138059 20410 leader_election.cc:336] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 67329cbbe18040088d9ac6c4ab899d58 (127.18.118.188:39171): Timed out: RequestConsensusVote RPC to 127.18.118.188:39171 timed out after 0.255s (SENT)
I20251023 08:00:16.158511 20902 leader_election.cc:290] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549), a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
W20251023 08:00:16.166458 20673 leader_election.cc:336] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549): Timed out: RequestConsensusVote RPC to 127.18.118.129:34549 timed out after 0.271s (SENT)
W20251023 08:00:16.216111 20671 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.005s	sys 0.006s
W20251023 08:00:16.216531 20671 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.005s	sys 0.006s
I20251023 08:00:16.219070 20630 raft_consensus.cc:3060] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 2 FOLLOWER]: Advancing to term 3
I20251023 08:00:16.234776 20913 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 08:00:16.234733 20630 raft_consensus.cc:1275] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 3 FOLLOWER]: Refusing update from remote peer 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20251023 08:00:16.237388 20626 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
I20251023 08:00:16.241529 20850 raft_consensus.cc:493] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 08:00:16.241537 20627 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
I20251023 08:00:16.242218 20850 raft_consensus.cc:515] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:16.243672 20914 raft_consensus.cc:493] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a5fa082ef7694b6a84e3a4c2add35a2e)
I20251023 08:00:16.244587 20914 raft_consensus.cc:515] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:16.245667 20627 raft_consensus.cc:2468] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in term 1.
I20251023 08:00:16.243665 20905 consensus_queue.cc:1048] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20251023 08:00:16.247443 20707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" is_pre_election: true
I20251023 08:00:16.249187 20914 leader_election.cc:290] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397), a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:16.249365 20844 raft_consensus.cc:493] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4fd4035d6b864683b312d9eba4de5a3e)
I20251023 08:00:16.249476 20785 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" is_pre_election: true
I20251023 08:00:16.249869 20844 raft_consensus.cc:515] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:16.251336 20581 leader_election.cc:304] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4fd4035d6b864683b312d9eba4de5a3e; no voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, a5fa082ef7694b6a84e3a4c2add35a2e
I20251023 08:00:16.251765 20844 leader_election.cc:290] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397), 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549)
I20251023 08:00:16.252275 20844 raft_consensus.cc:493] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 08:00:16.252243 20915 raft_consensus.cc:2749] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251023 08:00:16.252803 20844 raft_consensus.cc:515] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:16.254104 20502 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: false dest_uuid: "67329cbbe18040088d9ac6c4ab899d58" is_pre_election: true
I20251023 08:00:16.254961 20844 leader_election.cc:290] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549), 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397)
I20251023 08:00:16.255890 20502 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: false dest_uuid: "67329cbbe18040088d9ac6c4ab899d58"
I20251023 08:00:16.257582 20502 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 1 LEADER]: Stepping down as leader of term 1
I20251023 08:00:16.258013 20502 raft_consensus.cc:740] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 67329cbbe18040088d9ac6c4ab899d58, State: Running, Role: LEADER
I20251023 08:00:16.258882 20502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 11, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:16.259599 20502 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:16.266358 20501 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: false dest_uuid: "67329cbbe18040088d9ac6c4ab899d58" is_pre_election: true
I20251023 08:00:16.268079 20500 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: false dest_uuid: "67329cbbe18040088d9ac6c4ab899d58"
I20251023 08:00:16.270545 20502 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 2 FOLLOWER]: Refusing update from remote peer b303b3b1ff704ead975b9a3f0206f0d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 2 index: 12. (index mismatch)
I20251023 08:00:16.272646 20904 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20251023 08:00:16.273128 20707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" is_pre_election: true
I20251023 08:00:16.273784 20708 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" is_pre_election: true
W20251023 08:00:16.276229 20477 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20251023 08:00:16.278213 20903 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:16.279111 20903 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:16.283560 20898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:16.284400 20898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:16.285223 20370 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 2 FOLLOWER]: Rejecting Update request from peer 67329cbbe18040088d9ac6c4ab899d58 for earlier term 1. Current term is 2. Ops: []
I20251023 08:00:16.285365 20628 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
I20251023 08:00:16.286320 20502 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8080b9557fda47c0af876ca344e2477a" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: false dest_uuid: "67329cbbe18040088d9ac6c4ab899d58" is_pre_election: true
I20251023 08:00:16.287148 20785 raft_consensus.cc:1275] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 3 FOLLOWER]: Refusing update from remote peer 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20251023 08:00:16.287441 20500 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8080b9557fda47c0af876ca344e2477a" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: false dest_uuid: "67329cbbe18040088d9ac6c4ab899d58" is_pre_election: true
I20251023 08:00:16.288582 20784 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" is_pre_election: true
I20251023 08:00:16.289403 20902 consensus_queue.cc:1048] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 08:00:16.289968 20783 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" is_pre_election: true
I20251023 08:00:16.292320 20671 leader_election.cc:304] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb; no voters: 4fd4035d6b864683b312d9eba4de5a3e, a5fa082ef7694b6a84e3a4c2add35a2e
I20251023 08:00:16.293273 20902 raft_consensus.cc:2749] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20251023 08:00:16.294384 20671 leader_election.cc:304] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb; no voters: 4fd4035d6b864683b312d9eba4de5a3e, a5fa082ef7694b6a84e3a4c2add35a2e
I20251023 08:00:16.296482 20902 raft_consensus.cc:2749] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251023 08:00:16.296959 20913 catalog_manager.cc:679] Loaded metadata for table test-table [id=ebb365d398e748ffadc1298f5322a54f]
I20251023 08:00:16.297135 20627 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
I20251023 08:00:16.298735 20627 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
I20251023 08:00:16.299171 20751 leader_election.cc:304] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 4fd4035d6b864683b312d9eba4de5a3e, 7fdcd2a5f83a48cb8f77f5f4fd8929bb
I20251023 08:00:16.300235 20896 raft_consensus.cc:2749] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20251023 08:00:16.301342 20751 leader_election.cc:304] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 4fd4035d6b864683b312d9eba4de5a3e, 7fdcd2a5f83a48cb8f77f5f4fd8929bb
I20251023 08:00:16.302336 20896 raft_consensus.cc:2749] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20251023 08:00:16.317773 20907 sys_catalog.cc:455] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b303b3b1ff704ead975b9a3f0206f0d1. Latest consensus state: current_term: 2 leader_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:16.318871 20907 sys_catalog.cc:458] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:16.321214 20786 tablet_service.cc:2066] Received LeaderStepDown RPC: tablet_id: "14613f942a7b438bbf511fe370048d56"
dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e"
 from {username='slave'} at 127.0.0.1:50884
I20251023 08:00:16.321870 20786 raft_consensus.cc:591] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 1 LEADER]: Received request to step down
I20251023 08:00:16.322185 20786 raft_consensus.cc:3055] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 1 LEADER]: Stepping down as leader of term 1
I20251023 08:00:16.322475 20786 raft_consensus.cc:740] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a5fa082ef7694b6a84e3a4c2add35a2e, State: Running, Role: LEADER
I20251023 08:00:16.323175 20786 consensus_queue.cc:260] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:16.324062 20786 raft_consensus.cc:3060] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:16.334731 20629 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
I20251023 08:00:16.332839 20913 tablet_loader.cc:96] loaded metadata for tablet 0827630b93ca4fb8b649d9567669c1fe (table test-table [id=ebb365d398e748ffadc1298f5322a54f])
W20251023 08:00:16.347236 20672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.004s	sys 0.007s
W20251023 08:00:16.347612 20672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.004s	sys 0.007s
I20251023 08:00:16.355621 20889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b303b3b1ff704ead975b9a3f0206f0d1. Latest consensus state: current_term: 2 leader_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:16.356689 20889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:16.356706 20850 leader_election.cc:290] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397), a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:16.356920 20708 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" is_pre_election: true
I20251023 08:00:16.357813 20786 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" is_pre_election: true
I20251023 08:00:16.359165 20581 leader_election.cc:304] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4fd4035d6b864683b312d9eba4de5a3e; no voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, a5fa082ef7694b6a84e3a4c2add35a2e
I20251023 08:00:16.359997 20850 raft_consensus.cc:2749] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20251023 08:00:16.361670 20913 tablet_loader.cc:96] loaded metadata for tablet 14613f942a7b438bbf511fe370048d56 (table test-table [id=ebb365d398e748ffadc1298f5322a54f])
I20251023 08:00:16.362720 20913 tablet_loader.cc:96] loaded metadata for tablet 8fc078fb161a4da6b002c30d348ac289 (table test-table [id=ebb365d398e748ffadc1298f5322a54f])
I20251023 08:00:16.364045 20913 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 08:00:16.365371 20436 raft_consensus.cc:1240] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 2 LEADER]: Rejecting Update request from peer 67329cbbe18040088d9ac6c4ab899d58 for earlier term 1. Current term is 2. Ops: []
I20251023 08:00:16.368795 20913 catalog_manager.cc:1269] Loaded cluster ID: e45bbc9edac44b53bbefc3bb992dca7f
I20251023 08:00:16.369233 20913 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 08:00:16.368739 20907 sys_catalog.cc:455] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:16.369637 20907 sys_catalog.cc:458] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:16.373461 20913 catalog_manager.cc:1514] Loading token signing keys...
I20251023 08:00:16.376613 20913 catalog_manager.cc:6033] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1: Loaded TSK: 0
I20251023 08:00:16.377596 20913 catalog_manager.cc:1524] Initializing in-progress tserver states...
W20251023 08:00:16.396560 20392 debug-util.cc:398] Leaking SignalData structure 0x7b0800093280 after lost signal to thread 18909
W20251023 08:00:16.397523 20392 debug-util.cc:398] Leaking SignalData structure 0x7b0800061260 after lost signal to thread 20395
W20251023 08:00:16.398263 20392 debug-util.cc:398] Leaking SignalData structure 0x7b0800032ca0 after lost signal to thread 20460
W20251023 08:00:16.399042 20392 debug-util.cc:398] Leaking SignalData structure 0x7b0800248060 after lost signal to thread 20526
W20251023 08:00:16.400482 20392 debug-util.cc:398] Leaking SignalData structure 0x7b08002193c0 after lost signal to thread 20654
W20251023 08:00:16.401533 20392 debug-util.cc:398] Leaking SignalData structure 0x7b0800065540 after lost signal to thread 20732
W20251023 08:00:16.402662 20392 debug-util.cc:398] Leaking SignalData structure 0x7b0800223e40 after lost signal to thread 20810
I20251023 08:00:16.412760 20426 catalog_manager.cc:5649] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb reported cstate change: term changed from 2 to 3, leader changed from a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131) to 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130). New cstate: current_term: 3 leader_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } health_report { overall_health: HEALTHY } } }
I20251023 08:00:16.499792 20850 raft_consensus.cc:493] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a5fa082ef7694b6a84e3a4c2add35a2e)
I20251023 08:00:16.500550 20850 raft_consensus.cc:515] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:16.502517 20656 heartbeater.cc:499] Master 127.18.118.189:44091 was elected leader, sending a full tablet report...
I20251023 08:00:16.503854 20850 leader_election.cc:290] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397), a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:16.505311 20708 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" is_pre_election: true
I20251023 08:00:16.506059 20708 raft_consensus.cc:2468] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fd4035d6b864683b312d9eba4de5a3e in term 1.
I20251023 08:00:16.507495 20586 leader_election.cc:304] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fd4035d6b864683b312d9eba4de5a3e, 7fdcd2a5f83a48cb8f77f5f4fd8929bb; no voters: 
I20251023 08:00:16.510110 20786 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" is_pre_election: true
I20251023 08:00:16.511188 20786 raft_consensus.cc:2468] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fd4035d6b864683b312d9eba4de5a3e in term 2.
I20251023 08:00:16.516945 20850 raft_consensus.cc:2804] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 08:00:16.517488 20850 raft_consensus.cc:493] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Starting leader election (detected failure of leader a5fa082ef7694b6a84e3a4c2add35a2e)
I20251023 08:00:16.517999 20850 raft_consensus.cc:3060] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:16.529719 20812 heartbeater.cc:499] Master 127.18.118.189:44091 was elected leader, sending a full tablet report...
I20251023 08:00:16.530795 20734 heartbeater.cc:499] Master 127.18.118.189:44091 was elected leader, sending a full tablet report...
I20251023 08:00:16.537952 20850 raft_consensus.cc:515] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:16.546504 20708 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb"
I20251023 08:00:16.547224 20708 raft_consensus.cc:3060] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:16.554745 20708 raft_consensus.cc:2468] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fd4035d6b864683b312d9eba4de5a3e in term 2.
I20251023 08:00:16.556993 20783 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e"
I20251023 08:00:16.564989 20783 raft_consensus.cc:2468] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fd4035d6b864683b312d9eba4de5a3e in term 2.
I20251023 08:00:16.566253 20581 leader_election.cc:304] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4fd4035d6b864683b312d9eba4de5a3e, a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 
I20251023 08:00:16.567342 20900 raft_consensus.cc:2804] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 2 FOLLOWER]: Leader election won for term 2
I20251023 08:00:16.567842 20900 raft_consensus.cc:697] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 2 LEADER]: Becoming Leader. State: Replica: 4fd4035d6b864683b312d9eba4de5a3e, State: Running, Role: LEADER
I20251023 08:00:16.568661 20900 consensus_queue.cc:237] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:16.574579 20850 leader_election.cc:290] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 2 election: Requested vote from peers 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397), a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:16.593147 20426 catalog_manager.cc:5649] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e reported cstate change: term changed from 1 to 2, leader changed from a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131) to 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129). New cstate: current_term: 2 leader_uuid: "4fd4035d6b864683b312d9eba4de5a3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } health_report { overall_health: UNKNOWN } } }
I20251023 08:00:16.669922 20708 raft_consensus.cc:1275] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Refusing update from remote peer 4fd4035d6b864683b312d9eba4de5a3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 08:00:16.671553 20850 consensus_queue.cc:1048] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 08:00:16.694141 20425 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:50970:
table { table_id: "ebb365d398e748ffadc1298f5322a54f" table_name: "test-table" } new_table_name: "NewTableName"
I20251023 08:00:16.696673 20783 raft_consensus.cc:1275] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Refusing update from remote peer 4fd4035d6b864683b312d9eba4de5a3e: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20251023 08:00:16.705920 20915 consensus_queue.cc:1048] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20251023 08:00:16.735039 20425 catalog_manager.cc:5101] could not convert tablet's 8fc078fb161a4da6b002c30d348ac289 schema from PB: OK
I20251023 08:00:16.751927 18906 master.cc:561] Master@127.18.118.189:44091 shutting down...
I20251023 08:00:16.848392 20935 tablet.cc:1722] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20251023 08:00:16.850682 20940 tablet.cc:1722] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20251023 08:00:16.881819 20939 tablet.cc:1722] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20251023 08:00:16.893971 20941 tablet.cc:1722] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20251023 08:00:16.902001 20943 tablet.cc:1722] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20251023 08:00:16.916177 20935 tablet.cc:1722] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20251023 08:00:16.909430 20942 tablet.cc:1722] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20251023 08:00:16.921950 20937 tablet.cc:1722] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
W20251023 08:00:16.930711 20936 log.cc:927] Time spent T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e: Append to log took a long time: real 0.051s	user 0.002s	sys 0.000s
I20251023 08:00:16.940884 20941 tablet.cc:1722] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e: Alter schema from (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    PRIMARY KEY (key)
) version 1
I20251023 08:00:16.982309 18906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 2 LEADER]: Raft consensus shutting down.
I20251023 08:00:16.983469 18906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:16.983927 18906 tablet_replica.cc:333] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1: stopping tablet replica
I20251023 08:00:17.000237 18906 master.cc:583] Master@127.18.118.189:44091 shutdown complete.
I20251023 08:00:17.145253 20948 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b303b3b1ff704ead975b9a3f0206f0d1)
I20251023 08:00:17.146319 20948 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:17.148321 20948 leader_election.cc:290] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b303b3b1ff704ead975b9a3f0206f0d1 (127.18.118.189:44091), 67329cbbe18040088d9ac6c4ab899d58 (127.18.118.188:39171)
I20251023 08:00:17.150104 20500 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8080b9557fda47c0af876ca344e2477a" candidate_term: 3 candidate_status { last_received { term: 2 index: 15 } } ignore_live_leader: false dest_uuid: "67329cbbe18040088d9ac6c4ab899d58" is_pre_election: true
W20251023 08:00:17.155637 20347 leader_election.cc:336] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b303b3b1ff704ead975b9a3f0206f0d1 (127.18.118.189:44091): Network error: Client connection negotiation failed: client connection to 127.18.118.189:44091: connect: Connection refused (error 111)
I20251023 08:00:17.156262 20347 leader_election.cc:304] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8080b9557fda47c0af876ca344e2477a; no voters: 67329cbbe18040088d9ac6c4ab899d58, b303b3b1ff704ead975b9a3f0206f0d1
I20251023 08:00:17.157274 20948 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20251023 08:00:17.158735 20951 raft_consensus.cc:493] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b303b3b1ff704ead975b9a3f0206f0d1)
I20251023 08:00:17.159328 20951 raft_consensus.cc:515] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:17.161782 20951 leader_election.cc:290] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 8080b9557fda47c0af876ca344e2477a (127.18.118.190:38225), b303b3b1ff704ead975b9a3f0206f0d1 (127.18.118.189:44091)
I20251023 08:00:17.163304 20370 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67329cbbe18040088d9ac6c4ab899d58" candidate_term: 3 candidate_status { last_received { term: 2 index: 15 } } ignore_live_leader: false dest_uuid: "8080b9557fda47c0af876ca344e2477a" is_pre_election: true
I20251023 08:00:17.164040 20370 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67329cbbe18040088d9ac6c4ab899d58 in term 2.
I20251023 08:00:17.166973 20478 leader_election.cc:304] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67329cbbe18040088d9ac6c4ab899d58, 8080b9557fda47c0af876ca344e2477a; no voters: 
I20251023 08:00:17.167690 20951 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251023 08:00:17.168113 20951 raft_consensus.cc:493] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 2 FOLLOWER]: Starting leader election (detected failure of leader b303b3b1ff704ead975b9a3f0206f0d1)
I20251023 08:00:17.168483 20951 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 2 FOLLOWER]: Advancing to term 3
W20251023 08:00:17.168984 20478 leader_election.cc:336] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b303b3b1ff704ead975b9a3f0206f0d1 (127.18.118.189:44091): Network error: Client connection negotiation failed: client connection to 127.18.118.189:44091: connect: Connection refused (error 111)
I20251023 08:00:17.178485 20951 raft_consensus.cc:515] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:17.180689 20951 leader_election.cc:290] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [CANDIDATE]: Term 3 election: Requested vote from peers 8080b9557fda47c0af876ca344e2477a (127.18.118.190:38225), b303b3b1ff704ead975b9a3f0206f0d1 (127.18.118.189:44091)
I20251023 08:00:17.181392 20370 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67329cbbe18040088d9ac6c4ab899d58" candidate_term: 3 candidate_status { last_received { term: 2 index: 15 } } ignore_live_leader: false dest_uuid: "8080b9557fda47c0af876ca344e2477a"
I20251023 08:00:17.181980 20370 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 2 FOLLOWER]: Advancing to term 3
I20251023 08:00:17.189633 20370 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67329cbbe18040088d9ac6c4ab899d58 in term 3.
I20251023 08:00:17.195958 20478 leader_election.cc:304] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67329cbbe18040088d9ac6c4ab899d58, 8080b9557fda47c0af876ca344e2477a; no voters: 
W20251023 08:00:17.196949 20478 leader_election.cc:336] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b303b3b1ff704ead975b9a3f0206f0d1 (127.18.118.189:44091): Network error: Client connection negotiation failed: client connection to 127.18.118.189:44091: connect: Connection refused (error 111)
I20251023 08:00:17.196934 20951 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 3 FOLLOWER]: Leader election won for term 3
I20251023 08:00:17.200142 20951 raft_consensus.cc:697] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 3 LEADER]: Becoming Leader. State: Replica: 67329cbbe18040088d9ac6c4ab899d58, State: Running, Role: LEADER
I20251023 08:00:17.201450 20951 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 2.15, Last appended by leader: 15, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:17.207626 20954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67329cbbe18040088d9ac6c4ab899d58. Latest consensus state: current_term: 3 leader_uuid: "67329cbbe18040088d9ac6c4ab899d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:17.208585 20954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:17.210254 20955 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 08:00:17.216122 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251023 08:00:17.217662 20955 catalog_manager.cc:679] Loaded metadata for table NewTableName [id=ebb365d398e748ffadc1298f5322a54f]
I20251023 08:00:17.224224 20955 tablet_loader.cc:96] loaded metadata for tablet 0827630b93ca4fb8b649d9567669c1fe (table NewTableName [id=ebb365d398e748ffadc1298f5322a54f])
W20251023 08:00:17.224457 20956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:17.227839 20955 tablet_loader.cc:96] loaded metadata for tablet 14613f942a7b438bbf511fe370048d56 (table NewTableName [id=ebb365d398e748ffadc1298f5322a54f])
I20251023 08:00:17.229274 20955 tablet_loader.cc:96] loaded metadata for tablet 8fc078fb161a4da6b002c30d348ac289 (table NewTableName [id=ebb365d398e748ffadc1298f5322a54f])
I20251023 08:00:17.230749 20955 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 08:00:17.235684 20955 catalog_manager.cc:1269] Loaded cluster ID: e45bbc9edac44b53bbefc3bb992dca7f
I20251023 08:00:17.236002 20955 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
W20251023 08:00:17.235109 20957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:17.241762 20955 catalog_manager.cc:1514] Loading token signing keys...
I20251023 08:00:17.248234 20955 catalog_manager.cc:6033] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58: Loaded TSK: 0
I20251023 08:00:17.249387 20955 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 08:00:17.249596 18906 server_base.cc:1047] running on GCE node
W20251023 08:00:17.250475 20959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:17.251794 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:17.252130 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:17.252353 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206417252334 us; error 0 us; skew 500 ppm
I20251023 08:00:17.253226 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:17.256582 18906 webserver.cc:492] Webserver started at http://127.18.118.189:33889/ using document root <none> and password file <none>
I20251023 08:00:17.257169 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:17.257351 18906 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251023 08:00:17.261557 18906 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.001s	sys 0.002s
I20251023 08:00:17.265174 20964 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:17.266278 18906 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20251023 08:00:17.266659 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/master-1-root
uuid: "b303b3b1ff704ead975b9a3f0206f0d1"
format_stamp: "Formatted at 2025-10-23 08:00:12 on dist-test-slave-wkn7"
I20251023 08:00:17.267046 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20251023 08:00:17.289860 20478 consensus_peers.cc:597] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 -> Peer b303b3b1ff704ead975b9a3f0206f0d1 (127.18.118.189:44091): Couldn't send request to peer b303b3b1ff704ead975b9a3f0206f0d1. Status: Network error: Client connection negotiation failed: client connection to 127.18.118.189:44091: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 08:00:17.292388 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:17.293923 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:17.328697 20370 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 3 FOLLOWER]: Refusing update from remote peer 67329cbbe18040088d9ac6c4ab899d58: Log matching property violated. Preceding OpId in replica: term: 2 index: 15. Preceding OpId from leader: term: 3 index: 16. (index mismatch)
I20251023 08:00:17.331230 20954 consensus_queue.cc:1048] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20251023 08:00:17.338519 20948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67329cbbe18040088d9ac6c4ab899d58. Latest consensus state: current_term: 3 leader_uuid: "67329cbbe18040088d9ac6c4ab899d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:17.339378 20948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:17.347760 20951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "67329cbbe18040088d9ac6c4ab899d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:17.348666 20951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:17.361485 20948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "67329cbbe18040088d9ac6c4ab899d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:17.362344 20948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:17.374922 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.189:44091
I20251023 08:00:17.375139 21017 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.189:44091 every 8 connection(s)
I20251023 08:00:17.386942 21018 sys_catalog.cc:263] Verifying existing consensus state
I20251023 08:00:17.389822 20811 heartbeater.cc:499] Master 127.18.118.188:39171 was elected leader, sending a full tablet report...
I20251023 08:00:17.391937 20655 heartbeater.cc:499] Master 127.18.118.188:39171 was elected leader, sending a full tablet report...
I20251023 08:00:17.394085 20733 heartbeater.cc:499] Master 127.18.118.188:39171 was elected leader, sending a full tablet report...
I20251023 08:00:17.399185 21018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1: Bootstrap starting.
I20251023 08:00:17.489002 20812 heartbeater.cc:344] Connected to a master server at 127.18.118.189:44091
I20251023 08:00:17.493821 20734 heartbeater.cc:344] Connected to a master server at 127.18.118.189:44091
I20251023 08:00:17.503517 20492 catalog_manager.cc:6194] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58: NewTableName [id=ebb365d398e748ffadc1298f5322a54f] alter complete (version 1)
I20251023 08:00:17.508092 20656 heartbeater.cc:344] Connected to a master server at 127.18.118.189:44091
I20251023 08:00:17.550442 21018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=0 applied=15 ignored=0} inserts{seen=7 ignored=0} mutations{seen=11 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 08:00:17.551645 21018 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1: Bootstrap complete.
I20251023 08:00:17.554996 21018 raft_consensus.cc:359] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:17.555773 21018 raft_consensus.cc:740] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b303b3b1ff704ead975b9a3f0206f0d1, State: Initialized, Role: FOLLOWER
I20251023 08:00:17.556634 21018 consensus_queue.cc:260] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 2.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:17.558884 21027 sys_catalog.cc:455] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:17.559731 21027 sys_catalog.cc:458] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:17.560563 21018 sys_catalog.cc:565] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 08:00:17.560902 20991 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 2 FOLLOWER]: Advancing to term 3
I20251023 08:00:17.582276 21036 sys_catalog.cc:455] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "67329cbbe18040088d9ac6c4ab899d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:17.582573 21027 sys_catalog.cc:455] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67329cbbe18040088d9ac6c4ab899d58. Latest consensus state: current_term: 3 leader_uuid: "67329cbbe18040088d9ac6c4ab899d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:17.583468 21027 sys_catalog.cc:458] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:17.586517 20954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "67329cbbe18040088d9ac6c4ab899d58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:17.587177 20954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:17.588066 21036 sys_catalog.cc:458] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:17.614423 21042 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20251023 08:00:17.631441 20980 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" instance_seqno: 1761206413036099) as {username='slave'} at 127.0.0.1:51154; Asking this server to re-register.
I20251023 08:00:17.633215 20656 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:17.637320 20980 ts_manager.cc:194] Registered new tserver with Master: 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549)
I20251023 08:00:17.715471 20980 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" instance_seqno: 1761206413183980) as {username='slave'} at 127.0.0.1:51130; Asking this server to re-register.
I20251023 08:00:17.718778 20734 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:17.722751 20980 ts_manager.cc:194] Registered new tserver with Master: 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397)
I20251023 08:00:17.723268 20981 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" instance_seqno: 1761206413363196) as {username='slave'} at 127.0.0.1:51146; Asking this server to re-register.
I20251023 08:00:17.724725 20812 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:17.727706 20981 ts_manager.cc:194] Registered new tserver with Master: a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:17.727821 18906 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.348879284s
I20251023 08:00:18.619280 21042 catalog_manager.cc:1269] Loaded cluster ID: e45bbc9edac44b53bbefc3bb992dca7f
I20251023 08:00:18.619565 21042 catalog_manager.cc:1562] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1: loading cluster ID for follower catalog manager: success
I20251023 08:00:18.624285 21042 catalog_manager.cc:1584] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1: acquiring CA information for follower catalog manager: success
I20251023 08:00:18.627887 21042 catalog_manager.cc:1612] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20251023 08:00:20.030121 21047 raft_consensus.cc:493] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4fd4035d6b864683b312d9eba4de5a3e)
I20251023 08:00:20.030719 21047 raft_consensus.cc:515] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:20.033519 21047 leader_election.cc:290] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235), 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549)
I20251023 08:00:20.037114 21046 raft_consensus.cc:493] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 67329cbbe18040088d9ac6c4ab899d58)
I20251023 08:00:20.037712 21046 raft_consensus.cc:515] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:20.056198 21046 leader_election.cc:290] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8080b9557fda47c0af876ca344e2477a (127.18.118.190:38225), 67329cbbe18040088d9ac6c4ab899d58 (127.18.118.188:39171)
I20251023 08:00:20.060992 21048 raft_consensus.cc:493] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7fdcd2a5f83a48cb8f77f5f4fd8929bb)
I20251023 08:00:20.061575 21048 raft_consensus.cc:515] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:20.067009 21048 leader_election.cc:290] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397), a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:20.083433 21047 raft_consensus.cc:493] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4fd4035d6b864683b312d9eba4de5a3e)
I20251023 08:00:20.084007 21047 raft_consensus.cc:515] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:20.085830 21047 leader_election.cc:290] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549), a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:20.093972 20370 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "8080b9557fda47c0af876ca344e2477a" is_pre_election: true
I20251023 08:00:20.094655 20370 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b303b3b1ff704ead975b9a3f0206f0d1 in term 3.
I20251023 08:00:20.096197 20967 leader_election.cc:304] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8080b9557fda47c0af876ca344e2477a, b303b3b1ff704ead975b9a3f0206f0d1; no voters: 
I20251023 08:00:20.097046 21046 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20251023 08:00:20.097543 21046 raft_consensus.cc:493] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 67329cbbe18040088d9ac6c4ab899d58)
I20251023 08:00:20.097966 21046 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 3 FOLLOWER]: Advancing to term 4
I20251023 08:00:20.097877 21054 raft_consensus.cc:493] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4fd4035d6b864683b312d9eba4de5a3e)
I20251023 08:00:20.098604 21054 raft_consensus.cc:515] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:20.101084 21054 leader_election.cc:290] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549), 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397)
I20251023 08:00:20.104334 21046 raft_consensus.cc:515] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:20.106117 21054 raft_consensus.cc:493] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4fd4035d6b864683b312d9eba4de5a3e)
I20251023 08:00:20.106765 21054 raft_consensus.cc:515] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:20.107982 21046 leader_election.cc:290] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [CANDIDATE]: Term 4 election: Requested vote from peers 8080b9557fda47c0af876ca344e2477a (127.18.118.190:38225), 67329cbbe18040088d9ac6c4ab899d58 (127.18.118.188:39171)
I20251023 08:00:20.108507 20370 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "8080b9557fda47c0af876ca344e2477a"
I20251023 08:00:20.108690 21054 leader_election.cc:290] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397), 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549)
I20251023 08:00:20.109457 20370 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 3 FOLLOWER]: Advancing to term 4
I20251023 08:00:20.115974 20628 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
I20251023 08:00:20.116305 20370 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b303b3b1ff704ead975b9a3f0206f0d1 in term 4.
I20251023 08:00:20.117602 20967 leader_election.cc:304] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8080b9557fda47c0af876ca344e2477a, b303b3b1ff704ead975b9a3f0206f0d1; no voters: 
I20251023 08:00:20.118427 21046 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 4 FOLLOWER]: Leader election won for term 4
I20251023 08:00:20.121472 20704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" is_pre_election: true
I20251023 08:00:20.126681 21046 raft_consensus.cc:697] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 4 LEADER]: Becoming Leader. State: Replica: b303b3b1ff704ead975b9a3f0206f0d1, State: Running, Role: LEADER
I20251023 08:00:20.127630 21046 consensus_queue.cc:237] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 3.17, Last appended by leader: 17, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:20.131714 20628 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
I20251023 08:00:20.132668 20784 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" is_pre_election: true
I20251023 08:00:20.133333 20784 raft_consensus.cc:2468] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in term 2.
I20251023 08:00:20.131809 20783 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" is_pre_election: true
I20251023 08:00:20.134289 20783 raft_consensus.cc:2468] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in term 2.
I20251023 08:00:20.136026 20671 leader_election.cc:304] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 4fd4035d6b864683b312d9eba4de5a3e
I20251023 08:00:20.137238 20671 leader_election.cc:304] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 
I20251023 08:00:20.137861 21053 raft_consensus.cc:2804] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251023 08:00:20.138386 21053 raft_consensus.cc:493] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4fd4035d6b864683b312d9eba4de5a3e)
I20251023 08:00:20.138514 21047 raft_consensus.cc:2804] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251023 08:00:20.138907 21054 raft_consensus.cc:493] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7fdcd2a5f83a48cb8f77f5f4fd8929bb)
I20251023 08:00:20.138952 21047 raft_consensus.cc:493] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4fd4035d6b864683b312d9eba4de5a3e)
I20251023 08:00:20.139570 21047 raft_consensus.cc:3060] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Advancing to term 3
I20251023 08:00:20.139505 21054 raft_consensus.cc:515] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:20.143213 20628 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
W20251023 08:00:20.145881 20968 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.869s	user 0.009s	sys 0.003s
W20251023 08:00:20.146281 20968 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.010s	sys 0.003s
I20251023 08:00:20.147235 20704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" is_pre_election: true
I20251023 08:00:20.149583 20752 leader_election.cc:304] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 4fd4035d6b864683b312d9eba4de5a3e, 7fdcd2a5f83a48cb8f77f5f4fd8929bb
I20251023 08:00:20.150956 20370 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 4 FOLLOWER]: Rejecting Update request from peer 67329cbbe18040088d9ac6c4ab899d58 for earlier term 3. Current term is 4. Ops: []
I20251023 08:00:20.157541 21057 consensus_queue.cc:1059] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 }, Status: INVALID_TERM, Last received: 3.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20251023 08:00:20.158622 21057 raft_consensus.cc:3055] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 3 LEADER]: Stepping down as leader of term 3
I20251023 08:00:20.159083 21053 raft_consensus.cc:3060] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 2 FOLLOWER]: Advancing to term 3
I20251023 08:00:20.159107 21057 raft_consensus.cc:740] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 67329cbbe18040088d9ac6c4ab899d58, State: Running, Role: LEADER
I20251023 08:00:20.161278 21057 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 3.17, Last appended by leader: 17, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:20.162734 21057 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 3 FOLLOWER]: Advancing to term 4
I20251023 08:00:20.167618 20785 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" is_pre_election: true
I20251023 08:00:20.169494 20785 raft_consensus.cc:2468] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4fd4035d6b864683b312d9eba4de5a3e in term 3.
I20251023 08:00:20.171438 21047 raft_consensus.cc:515] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:20.178803 20627 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
I20251023 08:00:20.160724 20707 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" is_pre_election: true
I20251023 08:00:20.187347 20704 raft_consensus.cc:1240] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Rejecting Update request from peer 4fd4035d6b864683b312d9eba4de5a3e for earlier term 2. Current term is 3. Ops: []
I20251023 08:00:20.192677 20627 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e"
I20251023 08:00:20.194651 21047 leader_election.cc:290] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 3 election: Requested vote from peers a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235), 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549)
I20251023 08:00:20.195186 21054 leader_election.cc:290] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549), 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397)
I20251023 08:00:20.201409 21054 raft_consensus.cc:2749] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20251023 08:00:20.205741 21053 raft_consensus.cc:515] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:20.221458 20627 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e"
I20251023 08:00:20.224422 21053 leader_election.cc:290] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 3 election: Requested vote from peers 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549), a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:20.233752 20705 raft_consensus.cc:1240] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Rejecting Update request from peer 4fd4035d6b864683b312d9eba4de5a3e for earlier term 2. Current term is 3. Ops: []
I20251023 08:00:20.202924 20581 leader_election.cc:304] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4fd4035d6b864683b312d9eba4de5a3e, a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb
W20251023 08:00:20.248754 20581 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.118.131:44235, user_credentials={real_user=slave}} blocked reactor thread for 46103.1us
I20251023 08:00:20.250684 21052 raft_consensus.cc:2804] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 3 FOLLOWER]: Leader pre-election won for term 4
I20251023 08:00:20.250998 21052 raft_consensus.cc:493] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 3 FOLLOWER]: Starting leader election (detected failure of leader 7fdcd2a5f83a48cb8f77f5f4fd8929bb)
I20251023 08:00:20.251348 21052 raft_consensus.cc:3060] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 3 FOLLOWER]: Advancing to term 4
I20251023 08:00:20.255453 20785 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e"
I20251023 08:00:20.255714 20786 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e"
I20251023 08:00:20.257483 20671 leader_election.cc:304] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb; no voters: 4fd4035d6b864683b312d9eba4de5a3e, a5fa082ef7694b6a84e3a4c2add35a2e
I20251023 08:00:20.258829 20671 leader_election.cc:304] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb; no voters: 4fd4035d6b864683b312d9eba4de5a3e, a5fa082ef7694b6a84e3a4c2add35a2e
I20251023 08:00:20.262316 21053 raft_consensus.cc:2749] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20251023 08:00:20.263031 21047 raft_consensus.cc:2749] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20251023 08:00:20.265919 20629 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
W20251023 08:00:20.266906 20629 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35596 (request call id 54) took 214 ms (client timeout 271 ms). Trace:
W20251023 08:00:20.267351 20629 rpcz_store.cc:269] 1023 08:00:20.053017 (+     0us) service_pool.cc:168] Inserting onto call queue
1023 08:00:20.265430 (+212413us) service_pool.cc:225] Handling call
1023 08:00:20.266847 (+  1417us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20251023 08:00:20.281454 21060 consensus_queue.cc:1059] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
I20251023 08:00:20.281955 20991 raft_consensus.cc:1240] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 4 LEADER]: Rejecting Update request from peer 67329cbbe18040088d9ac6c4ab899d58 for earlier term 3. Current term is 4. Ops: []
W20251023 08:00:20.312543 20965 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: connection negotiation to 127.18.118.188:39171 for RPC RequestConsensusVote timed out after 0.247s (ON_OUTBOUND_QUEUE) [suppressed 14 similar messages]
W20251023 08:00:20.319303 20479 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20251023 08:00:20.309403 20752 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.118.130:38397, user_credentials={real_user=slave}} blocked reactor thread for 160033us
I20251023 08:00:20.325403 21062 consensus_queue.cc:1059] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20251023 08:00:20.342774 20586 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.118.130:38397, user_credentials={real_user=slave}} blocked reactor thread for 50596us
I20251023 08:00:20.355782 21052 raft_consensus.cc:515] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:20.357223 20629 raft_consensus.cc:1240] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 4 FOLLOWER]: Rejecting Update request from peer 7fdcd2a5f83a48cb8f77f5f4fd8929bb for earlier term 3. Current term is 4. Ops: []
I20251023 08:00:20.359720 20704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb"
I20251023 08:00:20.367817 21055 sys_catalog.cc:455] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b303b3b1ff704ead975b9a3f0206f0d1. Latest consensus state: current_term: 4 leader_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:20.368687 21055 sys_catalog.cc:458] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: This master's current role is: LEADER
W20251023 08:00:20.365804 20581 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.18.118.131:44235, user_credentials={real_user=slave}} blocked reactor thread for 93394.9us
W20251023 08:00:20.371222 20965 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.18.118.188:39171, user_credentials={real_user=slave}} blocked reactor thread for 58832.5us
I20251023 08:00:20.372733 20708 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" is_pre_election: true
I20251023 08:00:20.372627 21053 consensus_queue.cc:1059] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20251023 08:00:20.373576 20708 raft_consensus.cc:2393] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5fa082ef7694b6a84e3a4c2add35a2e in current term 3: Already voted for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in this term.
I20251023 08:00:20.375909 21057 raft_consensus.cc:493] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 08:00:20.376570 21057 raft_consensus.cc:515] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:20.379077 21057 leader_election.cc:290] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8080b9557fda47c0af876ca344e2477a (127.18.118.190:38225), b303b3b1ff704ead975b9a3f0206f0d1 (127.18.118.189:44091)
I20251023 08:00:20.381196 20370 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67329cbbe18040088d9ac6c4ab899d58" candidate_term: 5 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "8080b9557fda47c0af876ca344e2477a" is_pre_election: true
I20251023 08:00:20.383780 20370 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67329cbbe18040088d9ac6c4ab899d58 in term 4.
I20251023 08:00:20.385308 20478 leader_election.cc:304] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 67329cbbe18040088d9ac6c4ab899d58, 8080b9557fda47c0af876ca344e2477a; no voters: 
I20251023 08:00:20.386507 21056 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20251023 08:00:20.382591 20708 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:43716 (request call id 30) took 267 ms (client timeout 296 ms). Trace:
I20251023 08:00:20.386893 21056 raft_consensus.cc:493] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 08:00:20.387513 21056 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 4 FOLLOWER]: Advancing to term 5
I20251023 08:00:20.389611 21060 raft_consensus.cc:3055] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 2 LEADER]: Stepping down as leader of term 2
I20251023 08:00:20.390127 21060 raft_consensus.cc:740] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4fd4035d6b864683b312d9eba4de5a3e, State: Running, Role: LEADER
I20251023 08:00:20.391145 21060 consensus_queue.cc:260] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:20.393328 21060 raft_consensus.cc:3060] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 2 FOLLOWER]: Advancing to term 3
W20251023 08:00:20.387265 20708 rpcz_store.cc:269] 1023 08:00:20.106291 (+     0us) service_pool.cc:168] Inserting onto call queue
1023 08:00:20.370012 (+263721us) service_pool.cc:225] Handling call
1023 08:00:20.373854 (+  3842us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20251023 08:00:20.395939 21052 leader_election.cc:290] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 4 election: Requested vote from peers 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397), a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:20.396780 20752 leader_election.cc:304] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 4fd4035d6b864683b312d9eba4de5a3e, 7fdcd2a5f83a48cb8f77f5f4fd8929bb
I20251023 08:00:20.401074 21059 raft_consensus.cc:3060] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Advancing to term 3
I20251023 08:00:20.401858 20991 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67329cbbe18040088d9ac6c4ab899d58" candidate_term: 5 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" is_pre_election: true
I20251023 08:00:20.407402 20502 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "67329cbbe18040088d9ac6c4ab899d58" is_pre_election: true
I20251023 08:00:20.409027 21054 raft_consensus.cc:493] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 7fdcd2a5f83a48cb8f77f5f4fd8929bb)
I20251023 08:00:20.409066 20501 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "67329cbbe18040088d9ac6c4ab899d58"
I20251023 08:00:20.409591 21054 raft_consensus.cc:515] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:20.409941 21056 raft_consensus.cc:515] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:20.410575 20370 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 4 FOLLOWER]: Refusing update from remote peer b303b3b1ff704ead975b9a3f0206f0d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 17. Preceding OpId from leader: term: 4 index: 18. (index mismatch)
I20251023 08:00:20.411617 21054 leader_election.cc:290] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549), 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130:38397)
I20251023 08:00:20.411758 21056 leader_election.cc:290] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [CANDIDATE]: Term 5 election: Requested vote from peers 8080b9557fda47c0af876ca344e2477a (127.18.118.190:38225), b303b3b1ff704ead975b9a3f0206f0d1 (127.18.118.189:44091)
I20251023 08:00:20.413152 20708 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" is_pre_election: true
I20251023 08:00:20.413837 20500 raft_consensus.cc:1240] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 5 FOLLOWER]: Rejecting Update request from peer b303b3b1ff704ead975b9a3f0206f0d1 for earlier term 4. Current term is 5. Ops: []
I20251023 08:00:20.413926 20629 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
I20251023 08:00:20.414153 20991 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67329cbbe18040088d9ac6c4ab899d58" candidate_term: 5 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "b303b3b1ff704ead975b9a3f0206f0d1"
I20251023 08:00:20.413954 20708 raft_consensus.cc:2393] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a5fa082ef7694b6a84e3a4c2add35a2e in current term 3: Already voted for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in this term.
I20251023 08:00:20.413048 21046 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20251023 08:00:20.415596 20784 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "4fd4035d6b864683b312d9eba4de5a3e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e"
I20251023 08:00:20.416146 20784 raft_consensus.cc:3060] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 3 FOLLOWER]: Advancing to term 4
I20251023 08:00:20.420190 20370 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67329cbbe18040088d9ac6c4ab899d58" candidate_term: 5 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "8080b9557fda47c0af876ca344e2477a"
I20251023 08:00:20.422062 20478 leader_election.cc:304] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67329cbbe18040088d9ac6c4ab899d58; no voters: 8080b9557fda47c0af876ca344e2477a, b303b3b1ff704ead975b9a3f0206f0d1
I20251023 08:00:20.423508 21069 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 08:00:20.427000 20752 leader_election.cc:304] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 4fd4035d6b864683b312d9eba4de5a3e, 7fdcd2a5f83a48cb8f77f5f4fd8929bb
I20251023 08:00:20.429076 21054 raft_consensus.cc:3060] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 2 FOLLOWER]: Advancing to term 3
I20251023 08:00:20.432502 21064 raft_consensus.cc:3055] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 2 LEADER]: Stepping down as leader of term 2
I20251023 08:00:20.433007 21064 raft_consensus.cc:740] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4fd4035d6b864683b312d9eba4de5a3e, State: Running, Role: LEADER
I20251023 08:00:20.433857 21064 consensus_queue.cc:260] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:20.439177 21069 catalog_manager.cc:679] Loaded metadata for table NewTableName [id=ebb365d398e748ffadc1298f5322a54f]
I20251023 08:00:20.444183 21064 raft_consensus.cc:3060] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 2 FOLLOWER]: Advancing to term 3
I20251023 08:00:20.449707 21055 consensus_queue.cc:1059] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20251023 08:00:20.449958 20708 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0827630b93ca4fb8b649d9567669c1fe" candidate_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" is_pre_election: true
I20251023 08:00:20.450774 21055 raft_consensus.cc:3055] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 4 LEADER]: Stepping down as leader of term 4
I20251023 08:00:20.451200 21055 raft_consensus.cc:740] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b303b3b1ff704ead975b9a3f0206f0d1, State: Running, Role: LEADER
I20251023 08:00:20.451954 20752 leader_election.cc:304] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 4fd4035d6b864683b312d9eba4de5a3e, 7fdcd2a5f83a48cb8f77f5f4fd8929bb
I20251023 08:00:20.425334 21056 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20251023 08:00:20.424916 21053 raft_consensus.cc:3055] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 LEADER]: Stepping down as leader of term 3
I20251023 08:00:20.454020 21053 raft_consensus.cc:740] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, State: Running, Role: LEADER
I20251023 08:00:20.452126 21055 consensus_queue.cc:260] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 4.18, Last appended by leader: 18, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:20.467548 21054 raft_consensus.cc:2749] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20251023 08:00:20.466161 21055 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 4 FOLLOWER]: Advancing to term 5
I20251023 08:00:20.471015 21069 tablet_loader.cc:96] loaded metadata for tablet 0827630b93ca4fb8b649d9567669c1fe (table NewTableName [id=ebb365d398e748ffadc1298f5322a54f])
I20251023 08:00:20.470290 21053 consensus_queue.cc:260] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:20.472476 21053 raft_consensus.cc:3060] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Advancing to term 4
I20251023 08:00:20.473085 20784 raft_consensus.cc:2468] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4fd4035d6b864683b312d9eba4de5a3e in term 4.
I20251023 08:00:20.474072 20785 raft_consensus.cc:1240] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 4 FOLLOWER]: Rejecting Update request from peer 7fdcd2a5f83a48cb8f77f5f4fd8929bb for earlier term 3. Current term is 4. Ops: []
I20251023 08:00:20.461786 21059 raft_consensus.cc:2749] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20251023 08:00:20.475768 20786 raft_consensus.cc:1240] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 3 FOLLOWER]: Rejecting Update request from peer 4fd4035d6b864683b312d9eba4de5a3e for earlier term 2. Current term is 3. Ops: []
I20251023 08:00:20.475744 21054 raft_consensus.cc:2749] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20251023 08:00:20.476967 20581 leader_election.cc:304] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4fd4035d6b864683b312d9eba4de5a3e, a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb
W20251023 08:00:20.478714 20671 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20251023 08:00:20.478597 21063 raft_consensus.cc:2804] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 4 FOLLOWER]: Leader election won for term 4
I20251023 08:00:20.478847 21069 tablet_loader.cc:96] loaded metadata for tablet 14613f942a7b438bbf511fe370048d56 (table NewTableName [id=ebb365d398e748ffadc1298f5322a54f])
I20251023 08:00:20.479472 21063 raft_consensus.cc:697] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 4 LEADER]: Becoming Leader. State: Replica: 4fd4035d6b864683b312d9eba4de5a3e, State: Running, Role: LEADER
I20251023 08:00:20.480787 21069 tablet_loader.cc:96] loaded metadata for tablet 8fc078fb161a4da6b002c30d348ac289 (table NewTableName [id=ebb365d398e748ffadc1298f5322a54f])
I20251023 08:00:20.480443 21063 consensus_queue.cc:237] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:20.493819 21069 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 08:00:20.499389 21069 catalog_manager.cc:1269] Loaded cluster ID: e45bbc9edac44b53bbefc3bb992dca7f
I20251023 08:00:20.499809 21069 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 08:00:20.499195 21047 raft_consensus.cc:493] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 08:00:20.502652 21070 raft_consensus.cc:493] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20251023 08:00:20.503183 20967 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20251023 08:00:20.504289 21047 raft_consensus.cc:515] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:20.506731 21047 leader_election.cc:290] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235), 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549)
I20251023 08:00:20.507905 20629 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
I20251023 08:00:20.508466 20785 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" is_pre_election: true
I20251023 08:00:20.508579 20629 raft_consensus.cc:2468] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in term 3.
I20251023 08:00:20.509356 20785 raft_consensus.cc:2468] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in term 3.
I20251023 08:00:20.510959 20671 leader_election.cc:304] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 
I20251023 08:00:20.511890 21047 raft_consensus.cc:2804] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20251023 08:00:20.512269 21047 raft_consensus.cc:493] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 08:00:20.506886 21069 catalog_manager.cc:1514] Loading token signing keys...
I20251023 08:00:20.512624 21047 raft_consensus.cc:3060] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Advancing to term 4
I20251023 08:00:20.503367 21070 raft_consensus.cc:515] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:20.519551 21069 catalog_manager.cc:6033] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1: Loaded TSK: 0
I20251023 08:00:20.520992 21069 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 08:00:20.520565 21071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b303b3b1ff704ead975b9a3f0206f0d1. Latest consensus state: current_term: 4 leader_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:20.522194 21071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:20.524022 20785 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" is_pre_election: true
I20251023 08:00:20.524698 20785 raft_consensus.cc:2468] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in term 3.
I20251023 08:00:20.525869 20671 leader_election.cc:304] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 
I20251023 08:00:20.526726 21053 raft_consensus.cc:2804] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Leader pre-election won for term 4
I20251023 08:00:20.527104 21053 raft_consensus.cc:493] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 08:00:20.527463 21053 raft_consensus.cc:3060] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 3 FOLLOWER]: Advancing to term 4
I20251023 08:00:20.531394 20629 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e" is_pre_election: true
I20251023 08:00:20.532085 20629 raft_consensus.cc:2468] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in term 3.
I20251023 08:00:20.533344 21070 leader_election.cc:290] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549), a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:20.534582 21053 raft_consensus.cc:515] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:20.537314 20629 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e"
I20251023 08:00:20.537907 20785 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "14613f942a7b438bbf511fe370048d56" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e"
I20251023 08:00:20.537901 20629 raft_consensus.cc:3060] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 3 FOLLOWER]: Advancing to term 4
I20251023 08:00:20.538517 20785 raft_consensus.cc:3060] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 3 FOLLOWER]: Advancing to term 4
I20251023 08:00:20.539330 21053 leader_election.cc:290] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 4 election: Requested vote from peers 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549), a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235)
I20251023 08:00:20.540670 21047 raft_consensus.cc:515] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:20.543406 20786 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e"
I20251023 08:00:20.543993 20786 raft_consensus.cc:3060] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 3 FOLLOWER]: Advancing to term 4
I20251023 08:00:20.547145 20785 raft_consensus.cc:2468] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in term 4.
I20251023 08:00:20.548518 20671 leader_election.cc:304] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 
I20251023 08:00:20.549566 20629 raft_consensus.cc:2468] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in term 4.
I20251023 08:00:20.553138 21053 raft_consensus.cc:2804] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 4 FOLLOWER]: Leader election won for term 4
I20251023 08:00:20.554842 21053 raft_consensus.cc:697] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 4 LEADER]: Becoming Leader. State: Replica: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, State: Running, Role: LEADER
I20251023 08:00:20.556142 20629 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8fc078fb161a4da6b002c30d348ac289" candidate_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4fd4035d6b864683b312d9eba4de5a3e"
I20251023 08:00:20.555859 21053 consensus_queue.cc:237] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } }
I20251023 08:00:20.553154 20786 raft_consensus.cc:2468] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in term 4.
I20251023 08:00:20.558085 20671 leader_election.cc:304] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, a5fa082ef7694b6a84e3a4c2add35a2e; no voters: 
I20251023 08:00:20.554236 21047 leader_election.cc:290] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [CANDIDATE]: Term 4 election: Requested vote from peers a5fa082ef7694b6a84e3a4c2add35a2e (127.18.118.131:44235), 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549)
I20251023 08:00:20.559165 21070 raft_consensus.cc:2804] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 4 FOLLOWER]: Leader election won for term 4
I20251023 08:00:20.556777 20629 raft_consensus.cc:3060] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 3 FOLLOWER]: Advancing to term 4
I20251023 08:00:20.559852 21070 raft_consensus.cc:697] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 4 LEADER]: Becoming Leader. State: Replica: 7fdcd2a5f83a48cb8f77f5f4fd8929bb, State: Running, Role: LEADER
I20251023 08:00:20.560770 21070 consensus_queue.cc:237] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } }
I20251023 08:00:20.566386 20708 raft_consensus.cc:1275] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 4 FOLLOWER]: Refusing update from remote peer 4fd4035d6b864683b312d9eba4de5a3e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20251023 08:00:20.567730 20629 raft_consensus.cc:2468] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fdcd2a5f83a48cb8f77f5f4fd8929bb in term 4.
I20251023 08:00:20.568462 21063 consensus_queue.cc:1048] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20251023 08:00:20.577360 20786 raft_consensus.cc:1275] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 4 FOLLOWER]: Refusing update from remote peer 4fd4035d6b864683b312d9eba4de5a3e: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20251023 08:00:20.578907 21064 consensus_queue.cc:1048] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20251023 08:00:20.663689 21071 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b303b3b1ff704ead975b9a3f0206f0d1)
I20251023 08:00:20.664223 21071 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:20.666172 21071 leader_election.cc:290] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b303b3b1ff704ead975b9a3f0206f0d1 (127.18.118.189:44091), 67329cbbe18040088d9ac6c4ab899d58 (127.18.118.188:39171)
I20251023 08:00:20.669636 20785 raft_consensus.cc:1275] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 4 FOLLOWER]: Refusing update from remote peer 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20251023 08:00:20.671427 21070 consensus_queue.cc:1048] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 08:00:20.672539 20500 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8080b9557fda47c0af876ca344e2477a" candidate_term: 5 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: false dest_uuid: "67329cbbe18040088d9ac6c4ab899d58" is_pre_election: true
I20251023 08:00:20.674225 20500 raft_consensus.cc:2393] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8080b9557fda47c0af876ca344e2477a in current term 5: Already voted for candidate 67329cbbe18040088d9ac6c4ab899d58 in this term.
I20251023 08:00:20.698415 21055 raft_consensus.cc:493] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 08:00:20.699014 21055 raft_consensus.cc:515] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:20.702445 20371 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: false dest_uuid: "8080b9557fda47c0af876ca344e2477a" is_pre_election: true
I20251023 08:00:20.703074 20371 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b303b3b1ff704ead975b9a3f0206f0d1 in term 4.
I20251023 08:00:20.703715 21055 leader_election.cc:290] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8080b9557fda47c0af876ca344e2477a (127.18.118.190:38225), 67329cbbe18040088d9ac6c4ab899d58 (127.18.118.188:39171)
I20251023 08:00:20.704520 20967 leader_election.cc:304] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8080b9557fda47c0af876ca344e2477a, b303b3b1ff704ead975b9a3f0206f0d1; no voters: 
I20251023 08:00:20.711957 21056 raft_consensus.cc:493] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 08:00:20.714391 21056 raft_consensus.cc:515] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:20.716050 20784 raft_consensus.cc:1275] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 4 FOLLOWER]: Refusing update from remote peer 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20251023 08:00:20.716435 21056 leader_election.cc:290] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8080b9557fda47c0af876ca344e2477a (127.18.118.190:38225), b303b3b1ff704ead975b9a3f0206f0d1 (127.18.118.189:44091)
I20251023 08:00:20.724431 20500 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: false dest_uuid: "67329cbbe18040088d9ac6c4ab899d58" is_pre_election: true
I20251023 08:00:20.725179 20500 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b303b3b1ff704ead975b9a3f0206f0d1 in term 5.
I20251023 08:00:20.726408 21070 consensus_queue.cc:1048] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 08:00:20.727515 20991 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67329cbbe18040088d9ac6c4ab899d58" candidate_term: 6 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" is_pre_election: true
I20251023 08:00:20.727838 20371 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "67329cbbe18040088d9ac6c4ab899d58" candidate_term: 6 candidate_status { last_received { term: 3 index: 17 } } ignore_live_leader: false dest_uuid: "8080b9557fda47c0af876ca344e2477a" is_pre_election: true
I20251023 08:00:20.728446 20991 raft_consensus.cc:2410] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 67329cbbe18040088d9ac6c4ab899d58 for term 6 because replica has last-logged OpId of term: 4 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 17.
I20251023 08:00:20.730588 20629 raft_consensus.cc:1275] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 4 FOLLOWER]: Refusing update from remote peer 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20251023 08:00:20.731966 20627 raft_consensus.cc:1275] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 4 FOLLOWER]: Refusing update from remote peer 7fdcd2a5f83a48cb8f77f5f4fd8929bb: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20251023 08:00:20.728814 20371 raft_consensus.cc:2410] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 67329cbbe18040088d9ac6c4ab899d58 for term 6 because replica has last-logged OpId of term: 4 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 17.
I20251023 08:00:20.733574 20478 leader_election.cc:304] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 67329cbbe18040088d9ac6c4ab899d58; no voters: 8080b9557fda47c0af876ca344e2477a, b303b3b1ff704ead975b9a3f0206f0d1
I20251023 08:00:20.735289 21056 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20251023 08:00:20.738305 21055 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20251023 08:00:20.738795 21055 raft_consensus.cc:493] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 08:00:20.739168 21055 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 5 FOLLOWER]: Advancing to term 6
I20251023 08:00:20.743470 21053 consensus_queue.cc:1048] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20251023 08:00:20.743774 21047 consensus_queue.cc:1048] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251023 08:00:20.780567 21055 raft_consensus.cc:515] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:20.801481 20991 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8080b9557fda47c0af876ca344e2477a" candidate_term: 5 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: false dest_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" is_pre_election: true
I20251023 08:00:20.802430 20991 raft_consensus.cc:2368] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8080b9557fda47c0af876ca344e2477a for earlier term 5. Current term is 6.
I20251023 08:00:20.803879 20347 leader_election.cc:400] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [CANDIDATE]: Term 5 pre-election: Vote denied by peer b303b3b1ff704ead975b9a3f0206f0d1 (127.18.118.189:44091) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8080b9557fda47c0af876ca344e2477a for earlier term 5. Current term is 6.
I20251023 08:00:20.804219 20347 leader_election.cc:403] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20251023 08:00:20.806185 21071 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 4 FOLLOWER]: Advancing to term 6
I20251023 08:00:20.819131 21071 raft_consensus.cc:2749] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer b303b3b1ff704ead975b9a3f0206f0d1 (127.18.118.189:44091) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8080b9557fda47c0af876ca344e2477a for earlier term 5. Current term is 6.
I20251023 08:00:20.832954 20371 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: false dest_uuid: "8080b9557fda47c0af876ca344e2477a"
I20251023 08:00:20.834673 21055 leader_election.cc:290] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [CANDIDATE]: Term 6 election: Requested vote from peers 8080b9557fda47c0af876ca344e2477a (127.18.118.190:38225), 67329cbbe18040088d9ac6c4ab899d58 (127.18.118.188:39171)
I20251023 08:00:20.840197 20500 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 18 } } ignore_live_leader: false dest_uuid: "67329cbbe18040088d9ac6c4ab899d58"
I20251023 08:00:20.840891 20500 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 5 FOLLOWER]: Advancing to term 6
I20251023 08:00:20.846719 20371 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b303b3b1ff704ead975b9a3f0206f0d1 in term 6.
I20251023 08:00:20.852598 20967 leader_election.cc:304] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8080b9557fda47c0af876ca344e2477a, b303b3b1ff704ead975b9a3f0206f0d1; no voters: 
I20251023 08:00:20.853637 21055 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 6 FOLLOWER]: Leader election won for term 6
I20251023 08:00:20.854153 21055 raft_consensus.cc:697] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 6 LEADER]: Becoming Leader. State: Replica: b303b3b1ff704ead975b9a3f0206f0d1, State: Running, Role: LEADER
I20251023 08:00:20.855021 21055 consensus_queue.cc:237] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 4.18, Last appended by leader: 18, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } }
I20251023 08:00:20.859475 21046 sys_catalog.cc:455] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b303b3b1ff704ead975b9a3f0206f0d1. Latest consensus state: current_term: 6 leader_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:20.860297 21046 sys_catalog.cc:458] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:20.860975 21069 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 08:00:20.869233 21069 catalog_manager.cc:679] Loaded metadata for table NewTableName [id=ebb365d398e748ffadc1298f5322a54f]
I20251023 08:00:20.877981 21069 tablet_loader.cc:96] loaded metadata for tablet 0827630b93ca4fb8b649d9567669c1fe (table NewTableName [id=ebb365d398e748ffadc1298f5322a54f])
I20251023 08:00:20.878958 20500 raft_consensus.cc:2468] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b303b3b1ff704ead975b9a3f0206f0d1 in term 6.
I20251023 08:00:20.896037 21069 tablet_loader.cc:96] loaded metadata for tablet 14613f942a7b438bbf511fe370048d56 (table NewTableName [id=ebb365d398e748ffadc1298f5322a54f])
I20251023 08:00:20.898708 21069 tablet_loader.cc:96] loaded metadata for tablet 8fc078fb161a4da6b002c30d348ac289 (table NewTableName [id=ebb365d398e748ffadc1298f5322a54f])
I20251023 08:00:20.905632 21069 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 08:00:20.911388 21069 catalog_manager.cc:1269] Loaded cluster ID: e45bbc9edac44b53bbefc3bb992dca7f
I20251023 08:00:20.911890 21069 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 08:00:20.918223 21069 catalog_manager.cc:1514] Loading token signing keys...
I20251023 08:00:20.925472 21069 catalog_manager.cc:6033] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1: Loaded TSK: 0
I20251023 08:00:20.926498 21069 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 08:00:20.937685 20500 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 6 FOLLOWER]: Refusing update from remote peer b303b3b1ff704ead975b9a3f0206f0d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 17. Preceding OpId from leader: term: 6 index: 19. (index mismatch)
I20251023 08:00:20.939401 21055 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20251023 08:00:20.945365 21056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b303b3b1ff704ead975b9a3f0206f0d1. Latest consensus state: current_term: 6 leader_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:20.946215 21056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:20.956795 20371 raft_consensus.cc:1275] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 6 FOLLOWER]: Refusing update from remote peer b303b3b1ff704ead975b9a3f0206f0d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 18. Preceding OpId from leader: term: 6 index: 19. (index mismatch)
I20251023 08:00:20.956429 21055 sys_catalog.cc:455] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:20.958151 21055 sys_catalog.cc:458] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:20.959046 21055 consensus_queue.cc:1048] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20251023 08:00:20.974119 21056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:20.974967 21056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:20.977265 21093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:20.978098 21093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:21.011065 21095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:21.011335 21071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b303b3b1ff704ead975b9a3f0206f0d1. Latest consensus state: current_term: 6 leader_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:21.011931 21095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:21.011994 21071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:21.013955 21096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:21.014776 21096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [sys.catalog]: This master's current role is: FOLLOWER
I20251023 08:00:21.060425 21046 sys_catalog.cc:455] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8080b9557fda47c0af876ca344e2477a" member_type: VOTER last_known_addr { host: "127.18.118.190" port: 38225 } } peers { permanent_uuid: "b303b3b1ff704ead975b9a3f0206f0d1" member_type: VOTER last_known_addr { host: "127.18.118.189" port: 44091 } } peers { permanent_uuid: "67329cbbe18040088d9ac6c4ab899d58" member_type: VOTER last_known_addr { host: "127.18.118.188" port: 39171 } } }
I20251023 08:00:21.061309 21046 sys_catalog.cc:458] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:21.079706 20656 heartbeater.cc:499] Master 127.18.118.189:44091 was elected leader, sending a full tablet report...
I20251023 08:00:21.087453 20981 catalog_manager.cc:5649] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e reported cstate change: term changed from 3 to 4, leader changed from 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130) to 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129). New cstate: current_term: 4 leader_uuid: "4fd4035d6b864683b312d9eba4de5a3e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } health_report { overall_health: HEALTHY } } }
I20251023 08:00:21.089587 20981 catalog_manager.cc:5649] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e reported cstate change: term changed from 2 to 4, leader changed from 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129) to 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130). New cstate: current_term: 4 leader_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } }
I20251023 08:00:21.091132 20812 heartbeater.cc:499] Master 127.18.118.189:44091 was elected leader, sending a full tablet report...
I20251023 08:00:21.091213 20981 catalog_manager.cc:5649] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e reported cstate change: term changed from 2 to 4, leader changed from 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129) to 7fdcd2a5f83a48cb8f77f5f4fd8929bb (127.18.118.130). New cstate: current_term: 4 leader_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5fa082ef7694b6a84e3a4c2add35a2e" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 44235 } } peers { permanent_uuid: "7fdcd2a5f83a48cb8f77f5f4fd8929bb" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 38397 } } peers { permanent_uuid: "4fd4035d6b864683b312d9eba4de5a3e" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34549 } } }
I20251023 08:00:21.110999 20734 heartbeater.cc:499] Master 127.18.118.189:44091 was elected leader, sending a full tablet report...
I20251023 08:00:21.117053 20557 catalog_manager.cc:1612] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20251023 08:00:21.228178 20807 debug-util.cc:398] Leaking SignalData structure 0x7b0800211060 after lost signal to thread 18909
W20251023 08:00:21.229300 20807 debug-util.cc:398] Leaking SignalData structure 0x7b08002117c0 after lost signal to thread 20395
W20251023 08:00:21.230387 20807 debug-util.cc:398] Leaking SignalData structure 0x7b0800211e80 after lost signal to thread 20526
W20251023 08:00:21.231778 20807 debug-util.cc:398] Leaking SignalData structure 0x7b0800186b80 after lost signal to thread 20654
W20251023 08:00:21.232925 20807 debug-util.cc:398] Leaking SignalData structure 0x7b0800123600 after lost signal to thread 20732
W20251023 08:00:21.234095 20807 debug-util.cc:398] Leaking SignalData structure 0x7b0800062420 after lost signal to thread 20810
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/ts_tablet_manager-itest.cc:784: Failure
Expected equality of these values:
  live_row_count
    Which is: 802
  table_info->GetMetrics()->live_row_count->value()
    Which is: 1604
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/test_util.cc:403: Failure
Failed
Timed out waiting for assertion to pass.
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/ts_tablet_manager-itest.cc:768: Failure
Expected: check_function(table_infos[0].get(), live_row_count) doesn't generate new fatal failures in the current thread.
  Actual: it does.
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/ts_tablet_manager-itest.cc:786: Failure
Expected: GetLeaderMasterAndRun(live_row_count, [&] ( TableInfo* table_info, int64_t live_row_count) { do { AssertEventually([&] () { switch (0) case 0: default: if (const ::testing::AssertionResult gtest_ar_ = ::testing::AssertionResult(table_info->GetMetrics()->TableSupportsLiveRowCount())) ; else return ::testing::internal::AssertHelper(::testing::TestPartResult::kFatalFailure, "/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/ts_tablet_manager-itest.cc", 783, ::testing::internal::GetBoolAssertionFailureMessage( gtest_ar_, "table_info->GetMetrics()->TableSupportsLiveRowCount()", "false", "true") .c_str()) = ::testing::Message(); switch (0) case 0: default: if (const ::testing::AssertionResult gtest_ar = (::testing::internal::EqHelper::Compare("live_row_count", "table_info->GetMetrics()->live_row_count->value()", live_row_count, table_info->GetMetrics()->live_row_count->value()))) ; else return ::testing::internal::AssertHelper(::testing::TestPartResult::kFatalFailure, "/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/ts_tablet_manager-itest.cc", 784, gtest_ar.failure_message()) = ::testing::Message(); }); do { if (testing::Test::HasFatalFailure()) { return; } } while (0); } while (0); }) doesn't generate new fatal failures in the current thread.
  Actual: it does.
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/ts_tablet_manager-itest.cc:896: Failure
Expected: CheckStats(kRowsCount) doesn't generate new fatal failures in the current thread.
  Actual: it does.
I20251023 08:00:47.821589 18906 tablet_server.cc:178] TabletServer@127.18.118.129:0 shutting down...
W20251023 08:00:47.828727 20673 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 2 similar messages]
W20251023 08:00:47.840682 20673 consensus_peers.cc:597] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb -> Peer 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549): Couldn't send request to peer 4fd4035d6b864683b312d9eba4de5a3e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20251023 08:00:47.842104 20673 consensus_peers.cc:597] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb -> Peer 4fd4035d6b864683b312d9eba4de5a3e (127.18.118.129:34549): Couldn't send request to peer 4fd4035d6b864683b312d9eba4de5a3e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20251023 08:00:47.858978 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:47.859999 18906 tablet_replica.cc:333] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e: stopping tablet replica
I20251023 08:00:47.860986 18906 raft_consensus.cc:2243] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:47.861987 18906 raft_consensus.cc:2272] T 8fc078fb161a4da6b002c30d348ac289 P 4fd4035d6b864683b312d9eba4de5a3e [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:47.865335 18906 tablet_replica.cc:333] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e: stopping tablet replica
I20251023 08:00:47.865859 18906 raft_consensus.cc:2243] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:47.866304 18906 raft_consensus.cc:2272] T 14613f942a7b438bbf511fe370048d56 P 4fd4035d6b864683b312d9eba4de5a3e [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:47.869248 18906 tablet_replica.cc:333] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e: stopping tablet replica
I20251023 08:00:47.869742 18906 raft_consensus.cc:2243] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 4 LEADER]: Raft consensus shutting down.
I20251023 08:00:47.870568 18906 raft_consensus.cc:2272] T 0827630b93ca4fb8b649d9567669c1fe P 4fd4035d6b864683b312d9eba4de5a3e [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:47.894007 18906 tablet_server.cc:195] TabletServer@127.18.118.129:0 shutdown complete.
I20251023 08:00:47.908049 18906 tablet_server.cc:178] TabletServer@127.18.118.130:0 shutting down...
I20251023 08:00:47.955819 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:47.956552 18906 tablet_replica.cc:333] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: stopping tablet replica
I20251023 08:00:47.957350 18906 raft_consensus.cc:2243] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 4 LEADER]: Raft consensus shutting down.
I20251023 08:00:47.958585 18906 raft_consensus.cc:2272] T 8fc078fb161a4da6b002c30d348ac289 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:47.962136 18906 tablet_replica.cc:333] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: stopping tablet replica
I20251023 08:00:47.962615 18906 raft_consensus.cc:2243] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:47.963047 18906 raft_consensus.cc:2272] T 0827630b93ca4fb8b649d9567669c1fe P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:47.965883 18906 tablet_replica.cc:333] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb: stopping tablet replica
I20251023 08:00:47.966339 18906 raft_consensus.cc:2243] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 4 LEADER]: Raft consensus shutting down.
I20251023 08:00:47.967186 18906 raft_consensus.cc:2272] T 14613f942a7b438bbf511fe370048d56 P 7fdcd2a5f83a48cb8f77f5f4fd8929bb [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:47.993058 18906 tablet_server.cc:195] TabletServer@127.18.118.130:0 shutdown complete.
I20251023 08:00:48.007465 18906 tablet_server.cc:178] TabletServer@127.18.118.131:0 shutting down...
I20251023 08:00:48.040421 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:48.041247 18906 tablet_replica.cc:333] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e: stopping tablet replica
I20251023 08:00:48.041857 18906 raft_consensus.cc:2243] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:48.042526 18906 raft_consensus.cc:2272] T 0827630b93ca4fb8b649d9567669c1fe P a5fa082ef7694b6a84e3a4c2add35a2e [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:48.045408 18906 tablet_replica.cc:333] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e: stopping tablet replica
I20251023 08:00:48.045867 18906 raft_consensus.cc:2243] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:48.046312 18906 raft_consensus.cc:2272] T 8fc078fb161a4da6b002c30d348ac289 P a5fa082ef7694b6a84e3a4c2add35a2e [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:48.048748 18906 tablet_replica.cc:333] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e: stopping tablet replica
I20251023 08:00:48.049268 18906 raft_consensus.cc:2243] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 4 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:48.049800 18906 raft_consensus.cc:2272] T 14613f942a7b438bbf511fe370048d56 P a5fa082ef7694b6a84e3a4c2add35a2e [term 4 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:48.096824 18906 tablet_server.cc:195] TabletServer@127.18.118.131:0 shutdown complete.
I20251023 08:00:48.110162 18906 master.cc:561] Master@127.18.118.190:38225 shutting down...
I20251023 08:00:48.139822 18906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 6 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:48.140493 18906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a [term 6 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:48.140817 18906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8080b9557fda47c0af876ca344e2477a: stopping tablet replica
I20251023 08:00:48.183782 18906 master.cc:583] Master@127.18.118.190:38225 shutdown complete.
I20251023 08:00:48.199435 18906 master.cc:561] Master@127.18.118.189:44091 shutting down...
W20251023 08:00:48.214040 20967 consensus_peers.cc:597] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 -> Peer 8080b9557fda47c0af876ca344e2477a (127.18.118.190:38225): Couldn't send request to peer 8080b9557fda47c0af876ca344e2477a. Status: Network error: Client connection negotiation failed: client connection to 127.18.118.190:38225: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 08:00:48.228492 18906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 6 LEADER]: Raft consensus shutting down.
I20251023 08:00:48.229545 18906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b303b3b1ff704ead975b9a3f0206f0d1 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:48.229889 18906 tablet_replica.cc:333] stopping tablet replica
I20251023 08:00:48.252323 18906 master.cc:583] Master@127.18.118.189:44091 shutdown complete.
I20251023 08:00:48.269349 18906 master.cc:561] Master@127.18.118.188:39171 shutting down...
I20251023 08:00:48.290781 18906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 6 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:48.291347 18906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:48.291630 18906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 67329cbbe18040088d9ac6c4ab899d58: stopping tablet replica
I20251023 08:00:48.310581 18906 master.cc:583] Master@127.18.118.188:39171 shutdown complete.
I20251023 08:00:48.338570 18906 test_util.cc:183] -----------------------------------------------
I20251023 08:00:48.338833 18906 test_util.cc:184] Had failures, leaving test files at /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1761206396810293-18906-0
[  FAILED  ] TsTabletManagerITest.TestTableStats (35825 ms)
[ RUN      ] TsTabletManagerITest.TestDeleteTableDuringTabletCopy
I20251023 08:00:48.347676 18906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.18.118.190:41689
I20251023 08:00:48.348744 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:48.354341 21146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:48.354671 21147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:48.355995 21149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:48.358376 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:48.360271 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:48.360459 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:48.360584 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206448360572 us; error 0 us; skew 500 ppm
I20251023 08:00:48.361004 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:48.363298 18906 webserver.cc:492] Webserver started at http://127.18.118.190:38911/ using document root <none> and password file <none>
I20251023 08:00:48.363759 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:48.363907 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:48.364106 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:48.365238 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/master-0-root/instance:
uuid: "606bed4add164752982c205839508ffb"
format_stamp: "Formatted at 2025-10-23 08:00:48 on dist-test-slave-wkn7"
I20251023 08:00:48.369524 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20251023 08:00:48.372895 21154 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:48.373795 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20251023 08:00:48.374068 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/master-0-root
uuid: "606bed4add164752982c205839508ffb"
format_stamp: "Formatted at 2025-10-23 08:00:48 on dist-test-slave-wkn7"
I20251023 08:00:48.374325 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:48.401670 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:48.402897 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:48.441921 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.190:41689
I20251023 08:00:48.442032 21205 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.190:41689 every 8 connection(s)
I20251023 08:00:48.445839 21206 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:48.460068 21206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb: Bootstrap starting.
I20251023 08:00:48.464550 21206 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:48.468533 21206 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb: No bootstrap required, opened a new log
I20251023 08:00:48.470600 21206 raft_consensus.cc:359] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "606bed4add164752982c205839508ffb" member_type: VOTER }
I20251023 08:00:48.470983 21206 raft_consensus.cc:385] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:48.471201 21206 raft_consensus.cc:740] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 606bed4add164752982c205839508ffb, State: Initialized, Role: FOLLOWER
I20251023 08:00:48.471752 21206 consensus_queue.cc:260] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "606bed4add164752982c205839508ffb" member_type: VOTER }
I20251023 08:00:48.472201 21206 raft_consensus.cc:399] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 08:00:48.472424 21206 raft_consensus.cc:493] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 08:00:48.472681 21206 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:48.477138 21206 raft_consensus.cc:515] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "606bed4add164752982c205839508ffb" member_type: VOTER }
I20251023 08:00:48.477648 21206 leader_election.cc:304] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 606bed4add164752982c205839508ffb; no voters: 
I20251023 08:00:48.478721 21206 leader_election.cc:290] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb [CANDIDATE]: Term 1 election: Requested vote from peers 
I20251023 08:00:48.479099 21209 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:48.480324 21209 raft_consensus.cc:697] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb [term 1 LEADER]: Becoming Leader. State: Replica: 606bed4add164752982c205839508ffb, State: Running, Role: LEADER
I20251023 08:00:48.480998 21209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "606bed4add164752982c205839508ffb" member_type: VOTER }
I20251023 08:00:48.481714 21206 sys_catalog.cc:565] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb [sys.catalog]: configured and running, proceeding with master startup.
I20251023 08:00:48.485399 21210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "606bed4add164752982c205839508ffb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "606bed4add164752982c205839508ffb" member_type: VOTER } }
I20251023 08:00:48.486097 21210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:48.486163 21211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 606bed4add164752982c205839508ffb. Latest consensus state: current_term: 1 leader_uuid: "606bed4add164752982c205839508ffb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "606bed4add164752982c205839508ffb" member_type: VOTER } }
I20251023 08:00:48.486699 21211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:48.489285 21215 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 08:00:48.497615 21215 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 08:00:48.498611 18906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 08:00:48.507022 21215 catalog_manager.cc:1357] Generated new cluster ID: 35f44914e07e4f5686c2b18cce2b274a
I20251023 08:00:48.507373 21215 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 08:00:48.532905 21215 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 08:00:48.534228 21215 catalog_manager.cc:1514] Loading token signing keys...
I20251023 08:00:48.550731 21215 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb: Generated new TSK 0
I20251023 08:00:48.551324 21215 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 08:00:48.565166 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:48.570751 21227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:48.571861 21228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:48.573638 21230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:48.573696 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:48.574867 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:48.575054 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:48.575207 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206448575194 us; error 0 us; skew 500 ppm
I20251023 08:00:48.575717 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:48.578190 18906 webserver.cc:492] Webserver started at http://127.18.118.129:46651/ using document root <none> and password file <none>
I20251023 08:00:48.578697 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:48.578860 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:48.579123 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:48.580502 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/ts-0-root/instance:
uuid: "00305319b535416bba294b266d6469d9"
format_stamp: "Formatted at 2025-10-23 08:00:48 on dist-test-slave-wkn7"
I20251023 08:00:48.585173 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.007s	sys 0.000s
I20251023 08:00:48.589377 21235 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:48.590231 18906 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20251023 08:00:48.590623 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/ts-0-root
uuid: "00305319b535416bba294b266d6469d9"
format_stamp: "Formatted at 2025-10-23 08:00:48 on dist-test-slave-wkn7"
I20251023 08:00:48.591032 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:48.618306 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:48.619421 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:48.621021 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:00:48.624020 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:48.624189 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:48.624430 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:48.624588 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:48.665369 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.129:42511
I20251023 08:00:48.665488 21297 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.129:42511 every 8 connection(s)
I20251023 08:00:48.669824 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:48.677485 21302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:48.679771 21303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:48.680027 18906 server_base.cc:1047] running on GCE node
W20251023 08:00:48.681399 21305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:48.682395 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:48.682633 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:48.682809 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206448682789 us; error 0 us; skew 500 ppm
I20251023 08:00:48.683463 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:48.686226 18906 webserver.cc:492] Webserver started at http://127.18.118.130:43933/ using document root <none> and password file <none>
I20251023 08:00:48.686813 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:48.686820 21298 heartbeater.cc:344] Connected to a master server at 127.18.118.190:41689
I20251023 08:00:48.687191 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:48.687343 21298 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:48.687538 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:48.688191 21298 heartbeater.cc:507] Master 127.18.118.190:41689 requested a full tablet report, sending...
I20251023 08:00:48.688984 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/ts-1-root/instance:
uuid: "fb17728562f94c91a18c84e009767bec"
format_stamp: "Formatted at 2025-10-23 08:00:48 on dist-test-slave-wkn7"
I20251023 08:00:48.690503 21171 ts_manager.cc:194] Registered new tserver with Master: 00305319b535416bba294b266d6469d9 (127.18.118.129:42511)
I20251023 08:00:48.692183 21171 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55842
I20251023 08:00:48.693840 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20251023 08:00:48.697351 21310 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:48.698091 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20251023 08:00:48.698374 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/ts-1-root
uuid: "fb17728562f94c91a18c84e009767bec"
format_stamp: "Formatted at 2025-10-23 08:00:48 on dist-test-slave-wkn7"
I20251023 08:00:48.698652 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:48.720589 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:48.721879 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:48.723333 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:00:48.725728 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:48.725934 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:48.726166 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:48.726320 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:48.766464 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.130:40901
I20251023 08:00:48.766637 21372 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.130:40901 every 8 connection(s)
I20251023 08:00:48.771505 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:48.778512 21376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:48.779608 21377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:48.781394 21379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:48.783470 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:48.784344 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20251023 08:00:48.784349 21373 heartbeater.cc:344] Connected to a master server at 127.18.118.190:41689
W20251023 08:00:48.784662 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:48.784843 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206448784827 us; error 0 us; skew 500 ppm
I20251023 08:00:48.784826 21373 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:48.785521 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:48.785725 21373 heartbeater.cc:507] Master 127.18.118.190:41689 requested a full tablet report, sending...
I20251023 08:00:48.787601 21171 ts_manager.cc:194] Registered new tserver with Master: fb17728562f94c91a18c84e009767bec (127.18.118.130:40901)
I20251023 08:00:48.787973 18906 webserver.cc:492] Webserver started at http://127.18.118.131:44915/ using document root <none> and password file <none>
I20251023 08:00:48.788462 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:48.788651 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:48.788890 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:48.789067 21171 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55854
I20251023 08:00:48.790043 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/ts-2-root/instance:
uuid: "d3fbfcd3b08b46429f6e1182f2fb9c61"
format_stamp: "Formatted at 2025-10-23 08:00:48 on dist-test-slave-wkn7"
I20251023 08:00:48.794457 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20251023 08:00:48.797806 21384 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:48.798539 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20251023 08:00:48.798827 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/ts-2-root
uuid: "d3fbfcd3b08b46429f6e1182f2fb9c61"
format_stamp: "Formatted at 2025-10-23 08:00:48 on dist-test-slave-wkn7"
I20251023 08:00:48.799095 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:48.821661 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:48.822803 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:48.824237 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:00:48.826432 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:48.826634 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:48.826865 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:48.827021 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:48.865614 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.131:39483
I20251023 08:00:48.865708 21446 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.131:39483 every 8 connection(s)
I20251023 08:00:48.878847 21447 heartbeater.cc:344] Connected to a master server at 127.18.118.190:41689
I20251023 08:00:48.879245 21447 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:48.879969 21447 heartbeater.cc:507] Master 127.18.118.190:41689 requested a full tablet report, sending...
I20251023 08:00:48.881855 21171 ts_manager.cc:194] Registered new tserver with Master: d3fbfcd3b08b46429f6e1182f2fb9c61 (127.18.118.131:39483)
I20251023 08:00:48.882287 18906 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013613259s
I20251023 08:00:48.883240 21171 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55870
I20251023 08:00:48.916177 21171 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55878:
name: "test-table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
  }
}
W20251023 08:00:48.917886 21171 catalog_manager.cc:7011] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251023 08:00:48.958710 21263 tablet_service.cc:1505] Processing CreateTablet for tablet 6fee9e5c7545401184e0053ec6f9b73c (DEFAULT_TABLE table=test-table [id=2ccef331835946a8b8fd55b4d8d22a33]), partition=
I20251023 08:00:48.959339 21412 tablet_service.cc:1505] Processing CreateTablet for tablet 6fee9e5c7545401184e0053ec6f9b73c (DEFAULT_TABLE table=test-table [id=2ccef331835946a8b8fd55b4d8d22a33]), partition=
I20251023 08:00:48.959599 21338 tablet_service.cc:1505] Processing CreateTablet for tablet 6fee9e5c7545401184e0053ec6f9b73c (DEFAULT_TABLE table=test-table [id=2ccef331835946a8b8fd55b4d8d22a33]), partition=
I20251023 08:00:48.959936 21263 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fee9e5c7545401184e0053ec6f9b73c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:48.960434 21412 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fee9e5c7545401184e0053ec6f9b73c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:48.960566 21338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fee9e5c7545401184e0053ec6f9b73c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:48.972877 21463 tablet_bootstrap.cc:492] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9: Bootstrap starting.
I20251023 08:00:48.976956 21464 tablet_bootstrap.cc:492] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: Bootstrap starting.
I20251023 08:00:48.978607 21465 tablet_bootstrap.cc:492] T 6fee9e5c7545401184e0053ec6f9b73c P d3fbfcd3b08b46429f6e1182f2fb9c61: Bootstrap starting.
I20251023 08:00:48.979943 21463 tablet_bootstrap.cc:654] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:48.984843 21465 tablet_bootstrap.cc:654] T 6fee9e5c7545401184e0053ec6f9b73c P d3fbfcd3b08b46429f6e1182f2fb9c61: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:48.985052 21464 tablet_bootstrap.cc:654] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:48.987835 21463 tablet_bootstrap.cc:492] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9: No bootstrap required, opened a new log
I20251023 08:00:48.988409 21463 ts_tablet_manager.cc:1403] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.008s
I20251023 08:00:48.991434 21465 tablet_bootstrap.cc:492] T 6fee9e5c7545401184e0053ec6f9b73c P d3fbfcd3b08b46429f6e1182f2fb9c61: No bootstrap required, opened a new log
I20251023 08:00:48.991436 21464 tablet_bootstrap.cc:492] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: No bootstrap required, opened a new log
I20251023 08:00:48.992147 21465 ts_tablet_manager.cc:1403] T 6fee9e5c7545401184e0053ec6f9b73c P d3fbfcd3b08b46429f6e1182f2fb9c61: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.004s
I20251023 08:00:48.991518 21463 raft_consensus.cc:359] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00305319b535416bba294b266d6469d9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 42511 } } peers { permanent_uuid: "fb17728562f94c91a18c84e009767bec" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40901 } } peers { permanent_uuid: "d3fbfcd3b08b46429f6e1182f2fb9c61" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 39483 } }
I20251023 08:00:48.992566 21464 ts_tablet_manager.cc:1403] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: Time spent bootstrapping tablet: real 0.016s	user 0.011s	sys 0.004s
I20251023 08:00:48.992787 21463 raft_consensus.cc:385] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:48.993292 21463 raft_consensus.cc:740] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00305319b535416bba294b266d6469d9, State: Initialized, Role: FOLLOWER
I20251023 08:00:48.994118 21463 consensus_queue.cc:260] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00305319b535416bba294b266d6469d9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 42511 } } peers { permanent_uuid: "fb17728562f94c91a18c84e009767bec" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40901 } } peers { permanent_uuid: "d3fbfcd3b08b46429f6e1182f2fb9c61" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 39483 } }
I20251023 08:00:48.995023 21465 raft_consensus.cc:359] T 6fee9e5c7545401184e0053ec6f9b73c P d3fbfcd3b08b46429f6e1182f2fb9c61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00305319b535416bba294b266d6469d9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 42511 } } peers { permanent_uuid: "fb17728562f94c91a18c84e009767bec" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40901 } } peers { permanent_uuid: "d3fbfcd3b08b46429f6e1182f2fb9c61" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 39483 } }
I20251023 08:00:48.995463 21464 raft_consensus.cc:359] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00305319b535416bba294b266d6469d9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 42511 } } peers { permanent_uuid: "fb17728562f94c91a18c84e009767bec" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40901 } } peers { permanent_uuid: "d3fbfcd3b08b46429f6e1182f2fb9c61" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 39483 } }
I20251023 08:00:48.995911 21465 raft_consensus.cc:385] T 6fee9e5c7545401184e0053ec6f9b73c P d3fbfcd3b08b46429f6e1182f2fb9c61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:48.996076 21464 raft_consensus.cc:385] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:48.996356 21465 raft_consensus.cc:740] T 6fee9e5c7545401184e0053ec6f9b73c P d3fbfcd3b08b46429f6e1182f2fb9c61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3fbfcd3b08b46429f6e1182f2fb9c61, State: Initialized, Role: FOLLOWER
I20251023 08:00:48.996477 21464 raft_consensus.cc:740] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb17728562f94c91a18c84e009767bec, State: Initialized, Role: FOLLOWER
I20251023 08:00:48.997087 21298 heartbeater.cc:499] Master 127.18.118.190:41689 was elected leader, sending a full tablet report...
I20251023 08:00:48.997604 21463 ts_tablet_manager.cc:1434] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20251023 08:00:48.997349 21464 consensus_queue.cc:260] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00305319b535416bba294b266d6469d9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 42511 } } peers { permanent_uuid: "fb17728562f94c91a18c84e009767bec" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40901 } } peers { permanent_uuid: "d3fbfcd3b08b46429f6e1182f2fb9c61" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 39483 } }
W20251023 08:00:48.997954 21463 ts_tablet_manager.cc:1457] Injecting 50ms of latency into OpenTablet
I20251023 08:00:48.997471 21465 consensus_queue.cc:260] T 6fee9e5c7545401184e0053ec6f9b73c P d3fbfcd3b08b46429f6e1182f2fb9c61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00305319b535416bba294b266d6469d9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 42511 } } peers { permanent_uuid: "fb17728562f94c91a18c84e009767bec" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40901 } } peers { permanent_uuid: "d3fbfcd3b08b46429f6e1182f2fb9c61" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 39483 } }
I20251023 08:00:49.000170 21373 heartbeater.cc:499] Master 127.18.118.190:41689 was elected leader, sending a full tablet report...
I20251023 08:00:49.000892 21464 ts_tablet_manager.cc:1434] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
W20251023 08:00:49.001256 21464 ts_tablet_manager.cc:1457] Injecting 50ms of latency into OpenTablet
I20251023 08:00:49.003364 21447 heartbeater.cc:499] Master 127.18.118.190:41689 was elected leader, sending a full tablet report...
I20251023 08:00:49.004401 21465 ts_tablet_manager.cc:1434] T 6fee9e5c7545401184e0053ec6f9b73c P d3fbfcd3b08b46429f6e1182f2fb9c61: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
W20251023 08:00:49.004740 21465 ts_tablet_manager.cc:1457] Injecting 50ms of latency into OpenTablet
W20251023 08:00:49.020638 21374 tablet.cc:2378] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251023 08:00:49.219870 21469 raft_consensus.cc:493] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 08:00:49.220386 21469 raft_consensus.cc:515] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00305319b535416bba294b266d6469d9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 42511 } } peers { permanent_uuid: "fb17728562f94c91a18c84e009767bec" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40901 } } peers { permanent_uuid: "d3fbfcd3b08b46429f6e1182f2fb9c61" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 39483 } }
I20251023 08:00:49.222316 21469 leader_election.cc:290] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb17728562f94c91a18c84e009767bec (127.18.118.130:40901), d3fbfcd3b08b46429f6e1182f2fb9c61 (127.18.118.131:39483)
I20251023 08:00:49.233305 21422 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6fee9e5c7545401184e0053ec6f9b73c" candidate_uuid: "00305319b535416bba294b266d6469d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3fbfcd3b08b46429f6e1182f2fb9c61" is_pre_election: true
I20251023 08:00:49.234179 21422 raft_consensus.cc:2468] T 6fee9e5c7545401184e0053ec6f9b73c P d3fbfcd3b08b46429f6e1182f2fb9c61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00305319b535416bba294b266d6469d9 in term 0.
I20251023 08:00:49.234606 21348 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6fee9e5c7545401184e0053ec6f9b73c" candidate_uuid: "00305319b535416bba294b266d6469d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb17728562f94c91a18c84e009767bec" is_pre_election: true
I20251023 08:00:49.235272 21348 raft_consensus.cc:2468] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00305319b535416bba294b266d6469d9 in term 0.
I20251023 08:00:49.235462 21238 leader_election.cc:304] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00305319b535416bba294b266d6469d9, d3fbfcd3b08b46429f6e1182f2fb9c61; no voters: 
I20251023 08:00:49.236327 21469 raft_consensus.cc:2804] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 08:00:49.236616 21469 raft_consensus.cc:493] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 08:00:49.236874 21469 raft_consensus.cc:3060] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:49.241571 21469 raft_consensus.cc:515] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00305319b535416bba294b266d6469d9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 42511 } } peers { permanent_uuid: "fb17728562f94c91a18c84e009767bec" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40901 } } peers { permanent_uuid: "d3fbfcd3b08b46429f6e1182f2fb9c61" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 39483 } }
I20251023 08:00:49.243027 21469 leader_election.cc:290] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 [CANDIDATE]: Term 1 election: Requested vote from peers fb17728562f94c91a18c84e009767bec (127.18.118.130:40901), d3fbfcd3b08b46429f6e1182f2fb9c61 (127.18.118.131:39483)
I20251023 08:00:49.243925 21348 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6fee9e5c7545401184e0053ec6f9b73c" candidate_uuid: "00305319b535416bba294b266d6469d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb17728562f94c91a18c84e009767bec"
I20251023 08:00:49.243992 21422 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6fee9e5c7545401184e0053ec6f9b73c" candidate_uuid: "00305319b535416bba294b266d6469d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3fbfcd3b08b46429f6e1182f2fb9c61"
I20251023 08:00:49.244617 21348 raft_consensus.cc:3060] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:49.244679 21422 raft_consensus.cc:3060] T 6fee9e5c7545401184e0053ec6f9b73c P d3fbfcd3b08b46429f6e1182f2fb9c61 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:49.251822 21422 raft_consensus.cc:2468] T 6fee9e5c7545401184e0053ec6f9b73c P d3fbfcd3b08b46429f6e1182f2fb9c61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00305319b535416bba294b266d6469d9 in term 1.
I20251023 08:00:49.251829 21348 raft_consensus.cc:2468] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00305319b535416bba294b266d6469d9 in term 1.
I20251023 08:00:49.253242 21237 leader_election.cc:304] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00305319b535416bba294b266d6469d9, fb17728562f94c91a18c84e009767bec; no voters: 
I20251023 08:00:49.253890 21469 raft_consensus.cc:2804] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:49.254739 21469 raft_consensus.cc:697] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 [term 1 LEADER]: Becoming Leader. State: Replica: 00305319b535416bba294b266d6469d9, State: Running, Role: LEADER
I20251023 08:00:49.255445 21469 consensus_queue.cc:237] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00305319b535416bba294b266d6469d9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 42511 } } peers { permanent_uuid: "fb17728562f94c91a18c84e009767bec" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40901 } } peers { permanent_uuid: "d3fbfcd3b08b46429f6e1182f2fb9c61" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 39483 } }
I20251023 08:00:49.261611 21169 catalog_manager.cc:5649] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00305319b535416bba294b266d6469d9 (127.18.118.129). New cstate: current_term: 1 leader_uuid: "00305319b535416bba294b266d6469d9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00305319b535416bba294b266d6469d9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 42511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb17728562f94c91a18c84e009767bec" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d3fbfcd3b08b46429f6e1182f2fb9c61" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 39483 } health_report { overall_health: UNKNOWN } } }
I20251023 08:00:49.353863 21338 tablet_service.cc:1552] Processing DeleteTablet for tablet 6fee9e5c7545401184e0053ec6f9b73c with delete_type TABLET_DATA_TOMBSTONED from {username='slave'} at 127.0.0.1:50554
I20251023 08:00:49.355351 21477 tablet_replica.cc:333] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: stopping tablet replica
I20251023 08:00:49.355939 21477 raft_consensus.cc:2243] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:49.356344 21477 raft_consensus.cc:2272] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:49.358426 21477 ts_tablet_manager.cc:1916] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 08:00:49.368438 21477 ts_tablet_manager.cc:1929] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 0.0
I20251023 08:00:49.368808 21477 log.cc:1199] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: Deleting WAL directory at /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/ts-1-root/wals/6fee9e5c7545401184e0053ec6f9b73c
I20251023 08:00:49.373541 21478 ts_tablet_manager.cc:933] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: Initiating tablet copy from peer 00305319b535416bba294b266d6469d9 (127.18.118.129:42511)
I20251023 08:00:49.375272 21478 tablet_copy_client.cc:323] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: tablet copy: Beginning tablet copy session from remote peer at address 127.18.118.129:42511
I20251023 08:00:49.384720 21283 tablet_copy_service.cc:140] P 00305319b535416bba294b266d6469d9: Received BeginTabletCopySession request for tablet 6fee9e5c7545401184e0053ec6f9b73c from peer fb17728562f94c91a18c84e009767bec ({username='slave'} at 127.0.0.1:33960)
I20251023 08:00:49.385252 21283 tablet_copy_service.cc:161] P 00305319b535416bba294b266d6469d9: Beginning new tablet copy session on tablet 6fee9e5c7545401184e0053ec6f9b73c from peer fb17728562f94c91a18c84e009767bec at {username='slave'} at 127.0.0.1:33960: session id = fb17728562f94c91a18c84e009767bec-6fee9e5c7545401184e0053ec6f9b73c
I20251023 08:00:49.390923 21283 tablet_copy_source_session.cc:215] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 08:00:49.393543 21478 ts_tablet_manager.cc:1916] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: Deleting tablet data with delete state TABLET_DATA_COPYING
I20251023 08:00:49.403693 21478 ts_tablet_manager.cc:1929] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 0.0
I20251023 08:00:49.404297 21478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fee9e5c7545401184e0053ec6f9b73c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:49.409143 21478 tablet_copy_client.cc:806] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: tablet copy: Starting download of 0 data blocks...
I20251023 08:00:49.409628 21478 tablet_copy_client.cc:670] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: tablet copy: Starting download of 1 WAL segments...
I20251023 08:00:49.411144 21169 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55880:
table { table_id: "2ccef331835946a8b8fd55b4d8d22a33" table_name: "test-table" }
I20251023 08:00:49.411695 21169 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55880:
table { table_id: "2ccef331835946a8b8fd55b4d8d22a33" table_name: "test-table" }
I20251023 08:00:49.413290 21478 tablet_copy_client.cc:538] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 08:00:49.420385 21478 tablet_bootstrap.cc:492] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: Bootstrap starting.
I20251023 08:00:49.422003 21169 catalog_manager.cc:5936] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb: Sending DeleteTablet for 3 replicas of tablet 6fee9e5c7545401184e0053ec6f9b73c
I20251023 08:00:49.423820 21263 tablet_service.cc:1552] Processing DeleteTablet for tablet 6fee9e5c7545401184e0053ec6f9b73c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 08:00:49 UTC) from {username='slave'} at 127.0.0.1:33954
I20251023 08:00:49.424384 21338 tablet_service.cc:1552] Processing DeleteTablet for tablet 6fee9e5c7545401184e0053ec6f9b73c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 08:00:49 UTC) from {username='slave'} at 127.0.0.1:50550
I20251023 08:00:49.424834 21412 tablet_service.cc:1552] Processing DeleteTablet for tablet 6fee9e5c7545401184e0053ec6f9b73c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 08:00:49 UTC) from {username='slave'} at 127.0.0.1:50962
I20251023 08:00:49.425529 21481 tablet_replica.cc:333] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9: stopping tablet replica
I20251023 08:00:49.426328 21481 raft_consensus.cc:2243] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 [term 1 LEADER]: Raft consensus shutting down.
W20251023 08:00:49.426481 21156 catalog_manager.cc:4965] TS fb17728562f94c91a18c84e009767bec (127.18.118.130:40901): delete failed for tablet 6fee9e5c7545401184e0053ec6f9b73c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6fee9e5c7545401184e0053ec6f9b73c already in progress: copying tablet
I20251023 08:00:49.427249 21481 pending_rounds.cc:70] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9: Trying to abort 1 pending ops.
I20251023 08:00:49.427598 21481 pending_rounds.cc:77] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7213901616157556736 op_type: NO_OP noop_request { }
I20251023 08:00:49.428088 21482 tablet_replica.cc:333] T 6fee9e5c7545401184e0053ec6f9b73c P d3fbfcd3b08b46429f6e1182f2fb9c61: stopping tablet replica
I20251023 08:00:49.428083 21481 raft_consensus.cc:2889] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251023 08:00:49.428800 21481 raft_consensus.cc:2272] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:49.429201 21482 raft_consensus.cc:2243] T 6fee9e5c7545401184e0053ec6f9b73c P d3fbfcd3b08b46429f6e1182f2fb9c61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:49.429766 21482 raft_consensus.cc:2272] T 6fee9e5c7545401184e0053ec6f9b73c P d3fbfcd3b08b46429f6e1182f2fb9c61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:49.431979 21481 ts_tablet_manager.cc:1916] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 08:00:49.432421 21482 ts_tablet_manager.cc:1916] T 6fee9e5c7545401184e0053ec6f9b73c P d3fbfcd3b08b46429f6e1182f2fb9c61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 08:00:49.435184 21478 tablet_bootstrap.cc:492] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251023 08:00:49.436173 21478 tablet_bootstrap.cc:492] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: Bootstrap complete.
I20251023 08:00:49.436830 21478 ts_tablet_manager.cc:1403] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.002s
I20251023 08:00:49.439623 21478 raft_consensus.cc:359] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00305319b535416bba294b266d6469d9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 42511 } } peers { permanent_uuid: "fb17728562f94c91a18c84e009767bec" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40901 } } peers { permanent_uuid: "d3fbfcd3b08b46429f6e1182f2fb9c61" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 39483 } }
I20251023 08:00:49.440471 21478 raft_consensus.cc:740] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb17728562f94c91a18c84e009767bec, State: Initialized, Role: FOLLOWER
I20251023 08:00:49.441260 21478 consensus_queue.cc:260] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00305319b535416bba294b266d6469d9" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 42511 } } peers { permanent_uuid: "fb17728562f94c91a18c84e009767bec" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 40901 } } peers { permanent_uuid: "d3fbfcd3b08b46429f6e1182f2fb9c61" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 39483 } }
I20251023 08:00:49.442513 21481 ts_tablet_manager.cc:1929] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251023 08:00:49.442888 21481 log.cc:1199] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9: Deleting WAL directory at /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/ts-0-root/wals/6fee9e5c7545401184e0053ec6f9b73c
I20251023 08:00:49.443385 21478 ts_tablet_manager.cc:1434] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20251023 08:00:49.443560 21481 ts_tablet_manager.cc:1950] T 6fee9e5c7545401184e0053ec6f9b73c P 00305319b535416bba294b266d6469d9: Deleting consensus metadata
W20251023 08:00:49.443697 21478 ts_tablet_manager.cc:1457] Injecting 50ms of latency into OpenTablet
I20251023 08:00:49.445294 21155 catalog_manager.cc:4985] TS 00305319b535416bba294b266d6469d9 (127.18.118.129:42511): tablet 6fee9e5c7545401184e0053ec6f9b73c (table test-table [id=2ccef331835946a8b8fd55b4d8d22a33]) successfully deleted
I20251023 08:00:49.447340 21482 ts_tablet_manager.cc:1929] T 6fee9e5c7545401184e0053ec6f9b73c P d3fbfcd3b08b46429f6e1182f2fb9c61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20251023 08:00:49.447751 21482 log.cc:1199] T 6fee9e5c7545401184e0053ec6f9b73c P d3fbfcd3b08b46429f6e1182f2fb9c61: Deleting WAL directory at /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/ts-2-root/wals/6fee9e5c7545401184e0053ec6f9b73c
I20251023 08:00:49.448402 21482 ts_tablet_manager.cc:1950] T 6fee9e5c7545401184e0053ec6f9b73c P d3fbfcd3b08b46429f6e1182f2fb9c61: Deleting consensus metadata
I20251023 08:00:49.448510 21338 tablet_service.cc:1552] Processing DeleteTablet for tablet 6fee9e5c7545401184e0053ec6f9b73c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 08:00:49 UTC) from {username='slave'} at 127.0.0.1:50550
W20251023 08:00:49.449898 21156 catalog_manager.cc:4965] TS fb17728562f94c91a18c84e009767bec (127.18.118.130:40901): delete failed for tablet 6fee9e5c7545401184e0053ec6f9b73c because tablet deleting was already in progress. No further retry: Already present: State transition of tablet 6fee9e5c7545401184e0053ec6f9b73c already in progress: copying tablet
I20251023 08:00:49.450947 21157 catalog_manager.cc:4985] TS d3fbfcd3b08b46429f6e1182f2fb9c61 (127.18.118.131:39483): tablet 6fee9e5c7545401184e0053ec6f9b73c (table test-table [id=2ccef331835946a8b8fd55b4d8d22a33]) successfully deleted
I20251023 08:00:49.495743 21283 tablet_copy_service.cc:342] P 00305319b535416bba294b266d6469d9: Request end of tablet copy session fb17728562f94c91a18c84e009767bec-6fee9e5c7545401184e0053ec6f9b73c received from {username='slave'} at 127.0.0.1:33960
I20251023 08:00:49.496273 21283 tablet_copy_service.cc:434] P 00305319b535416bba294b266d6469d9: ending tablet copy session fb17728562f94c91a18c84e009767bec-6fee9e5c7545401184e0053ec6f9b73c on tablet 6fee9e5c7545401184e0053ec6f9b73c with peer fb17728562f94c91a18c84e009767bec
I20251023 08:00:49.499190 21338 tablet_service.cc:1552] Processing DeleteTablet for tablet 6fee9e5c7545401184e0053ec6f9b73c with delete_type TABLET_DATA_DELETED (Table deleted at 2025-10-23 08:00:49 UTC) from {username='slave'} at 127.0.0.1:50550
I20251023 08:00:49.500017 21477 tablet_replica.cc:333] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: stopping tablet replica
I20251023 08:00:49.500780 21477 raft_consensus.cc:2243] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:49.501282 21477 pending_rounds.cc:70] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: Trying to abort 1 pending ops.
I20251023 08:00:49.501538 21477 pending_rounds.cc:77] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7213901616157556736 op_type: NO_OP noop_request { }
I20251023 08:00:49.501915 21477 raft_consensus.cc:2889] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20251023 08:00:49.502286 21477 raft_consensus.cc:2272] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:49.504333 21477 ts_tablet_manager.cc:1916] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251023 08:00:49.514848 21477 ts_tablet_manager.cc:1929] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20251023 08:00:49.515156 21477 log.cc:1199] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: Deleting WAL directory at /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1761206396810293-18906-0/minicluster-data/ts-1-root/wals/6fee9e5c7545401184e0053ec6f9b73c
I20251023 08:00:49.515708 21477 ts_tablet_manager.cc:1950] T 6fee9e5c7545401184e0053ec6f9b73c P fb17728562f94c91a18c84e009767bec: Deleting consensus metadata
I20251023 08:00:49.518057 21156 catalog_manager.cc:4985] TS fb17728562f94c91a18c84e009767bec (127.18.118.130:40901): tablet 6fee9e5c7545401184e0053ec6f9b73c (table test-table [id=2ccef331835946a8b8fd55b4d8d22a33]) successfully deleted
I20251023 08:00:49.562424 18906 tablet_server.cc:178] TabletServer@127.18.118.129:0 shutting down...
I20251023 08:00:49.579975 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:49.586905 18906 tablet_server.cc:195] TabletServer@127.18.118.129:0 shutdown complete.
I20251023 08:00:49.595620 18906 tablet_server.cc:178] TabletServer@127.18.118.130:0 shutting down...
I20251023 08:00:49.611249 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:49.627645 18906 tablet_server.cc:195] TabletServer@127.18.118.130:0 shutdown complete.
I20251023 08:00:49.636281 18906 tablet_server.cc:178] TabletServer@127.18.118.131:0 shutting down...
I20251023 08:00:49.652457 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:49.668493 18906 tablet_server.cc:195] TabletServer@127.18.118.131:0 shutdown complete.
I20251023 08:00:49.676708 18906 master.cc:561] Master@127.18.118.190:41689 shutting down...
I20251023 08:00:49.691710 18906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb [term 1 LEADER]: Raft consensus shutting down.
I20251023 08:00:49.692256 18906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:49.692548 18906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 606bed4add164752982c205839508ffb: stopping tablet replica
I20251023 08:00:49.701676 18906 master.cc:583] Master@127.18.118.190:41689 shutdown complete.
[       OK ] TsTabletManagerITest.TestDeleteTableDuringTabletCopy (1379 ms)
[----------] 4 tests from TsTabletManagerITest (41919 ms total)

[----------] 4 tests from TxnStatusTabletManagementTest
[ RUN      ] TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet
I20251023 08:00:49.726508 18906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.18.118.190:35937
I20251023 08:00:49.727733 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:49.732277 21488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:49.733367 21489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:49.734922 18906 server_base.cc:1047] running on GCE node
W20251023 08:00:49.735642 21491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:49.736363 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:49.736558 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:49.736706 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206449736688 us; error 0 us; skew 500 ppm
I20251023 08:00:49.737229 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:49.739476 18906 webserver.cc:492] Webserver started at http://127.18.118.190:38533/ using document root <none> and password file <none>
I20251023 08:00:49.739967 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:49.740159 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:49.740408 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:49.741559 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/master-0-root/instance:
uuid: "cdfcc0a709f940c6a271b3232acfe885"
format_stamp: "Formatted at 2025-10-23 08:00:49 on dist-test-slave-wkn7"
I20251023 08:00:49.745728 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20251023 08:00:49.748838 21496 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:49.749616 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20251023 08:00:49.749890 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/master-0-root
uuid: "cdfcc0a709f940c6a271b3232acfe885"
format_stamp: "Formatted at 2025-10-23 08:00:49 on dist-test-slave-wkn7"
I20251023 08:00:49.750174 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:49.768797 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:49.769861 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:49.806770 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.190:35937
I20251023 08:00:49.806874 21547 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.190:35937 every 8 connection(s)
I20251023 08:00:49.810698 21548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:49.820940 21548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885: Bootstrap starting.
I20251023 08:00:49.825194 21548 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:49.828933 21548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885: No bootstrap required, opened a new log
I20251023 08:00:49.830816 21548 raft_consensus.cc:359] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdfcc0a709f940c6a271b3232acfe885" member_type: VOTER }
I20251023 08:00:49.831177 21548 raft_consensus.cc:385] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:49.831627 21548 raft_consensus.cc:740] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdfcc0a709f940c6a271b3232acfe885, State: Initialized, Role: FOLLOWER
I20251023 08:00:49.832347 21548 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdfcc0a709f940c6a271b3232acfe885" member_type: VOTER }
I20251023 08:00:49.832803 21548 raft_consensus.cc:399] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 08:00:49.832995 21548 raft_consensus.cc:493] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 08:00:49.833266 21548 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:49.837531 21548 raft_consensus.cc:515] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdfcc0a709f940c6a271b3232acfe885" member_type: VOTER }
I20251023 08:00:49.838016 21548 leader_election.cc:304] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdfcc0a709f940c6a271b3232acfe885; no voters: 
I20251023 08:00:49.839082 21548 leader_election.cc:290] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20251023 08:00:49.839407 21551 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:49.840780 21551 raft_consensus.cc:697] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885 [term 1 LEADER]: Becoming Leader. State: Replica: cdfcc0a709f940c6a271b3232acfe885, State: Running, Role: LEADER
I20251023 08:00:49.841452 21551 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdfcc0a709f940c6a271b3232acfe885" member_type: VOTER }
I20251023 08:00:49.841950 21548 sys_catalog.cc:565] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 08:00:49.844667 21553 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdfcc0a709f940c6a271b3232acfe885. Latest consensus state: current_term: 1 leader_uuid: "cdfcc0a709f940c6a271b3232acfe885" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdfcc0a709f940c6a271b3232acfe885" member_type: VOTER } }
I20251023 08:00:49.844587 21552 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cdfcc0a709f940c6a271b3232acfe885" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdfcc0a709f940c6a271b3232acfe885" member_type: VOTER } }
I20251023 08:00:49.845431 21553 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:49.845781 21552 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:49.848793 21557 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 08:00:49.856998 21557 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 08:00:49.857975 18906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 08:00:49.865764 21557 catalog_manager.cc:1357] Generated new cluster ID: b5fde508dd2d4eaeb9178dc4fe6c67a7
I20251023 08:00:49.866051 21557 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 08:00:49.883226 21557 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 08:00:49.884526 21557 catalog_manager.cc:1514] Loading token signing keys...
I20251023 08:00:49.896221 21557 catalog_manager.cc:6022] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885: Generated new TSK 0
I20251023 08:00:49.896965 21557 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 08:00:49.924996 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:49.930482 21569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:49.931460 21570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:49.934003 18906 server_base.cc:1047] running on GCE node
W20251023 08:00:49.934036 21572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:49.935034 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:49.935245 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:49.935396 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206449935379 us; error 0 us; skew 500 ppm
I20251023 08:00:49.935921 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:49.938644 18906 webserver.cc:492] Webserver started at http://127.18.118.129:42681/ using document root <none> and password file <none>
I20251023 08:00:49.939162 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:49.939352 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:49.939606 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:49.940692 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-0-root/instance:
uuid: "0106efdb5d174dcbb762451807bb5be7"
format_stamp: "Formatted at 2025-10-23 08:00:49 on dist-test-slave-wkn7"
I20251023 08:00:49.945457 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.001s
I20251023 08:00:49.948833 21577 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:49.949702 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20251023 08:00:49.949954 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-0-root
uuid: "0106efdb5d174dcbb762451807bb5be7"
format_stamp: "Formatted at 2025-10-23 08:00:49 on dist-test-slave-wkn7"
I20251023 08:00:49.950209 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:49.969126 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:49.970162 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:49.974838 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:49.975143 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:49.975450 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:49.975672 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:50.029793 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.129:38735
I20251023 08:00:50.029898 21647 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.129:38735 every 8 connection(s)
I20251023 08:00:50.034246 21648 heartbeater.cc:344] Connected to a master server at 127.18.118.190:35937
I20251023 08:00:50.034575 21648 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:50.035259 21648 heartbeater.cc:507] Master 127.18.118.190:35937 requested a full tablet report, sending...
I20251023 08:00:50.037055 21513 ts_manager.cc:194] Registered new tserver with Master: 0106efdb5d174dcbb762451807bb5be7 (127.18.118.129:38735)
I20251023 08:00:50.037303 18906 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.00446005s
I20251023 08:00:50.039455 21513 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54138
I20251023 08:00:50.071818 21613 tablet_service.cc:1505] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20251023 08:00:50.073143 21613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:50.084721 21658 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7: Bootstrap starting.
I20251023 08:00:50.089243 21658 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:50.093163 21658 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7: No bootstrap required, opened a new log
I20251023 08:00:50.093556 21658 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7: Time spent bootstrapping tablet: real 0.009s	user 0.008s	sys 0.000s
I20251023 08:00:50.095364 21658 raft_consensus.cc:359] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "0106efdb5d174dcbb762451807bb5be7" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38735 } }
I20251023 08:00:50.095742 21658 raft_consensus.cc:385] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:50.096007 21658 raft_consensus.cc:740] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0106efdb5d174dcbb762451807bb5be7, State: Initialized, Role: FOLLOWER
I20251023 08:00:50.096582 21658 consensus_queue.cc:260] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "0106efdb5d174dcbb762451807bb5be7" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38735 } }
I20251023 08:00:50.097045 21658 raft_consensus.cc:399] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 08:00:50.097343 21658 raft_consensus.cc:493] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 08:00:50.097610 21658 raft_consensus.cc:3060] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:50.102310 21658 raft_consensus.cc:515] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "0106efdb5d174dcbb762451807bb5be7" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38735 } }
I20251023 08:00:50.102838 21658 leader_election.cc:304] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0106efdb5d174dcbb762451807bb5be7; no voters: 
I20251023 08:00:50.104100 21658 leader_election.cc:290] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20251023 08:00:50.104415 21660 raft_consensus.cc:2804] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:50.105971 21648 heartbeater.cc:499] Master 127.18.118.190:35937 was elected leader, sending a full tablet report...
I20251023 08:00:50.106312 21660 raft_consensus.cc:697] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 1 LEADER]: Becoming Leader. State: Replica: 0106efdb5d174dcbb762451807bb5be7, State: Running, Role: LEADER
I20251023 08:00:50.106472 21658 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7: Time spent starting tablet: real 0.013s	user 0.011s	sys 0.001s
I20251023 08:00:50.106986 21660 consensus_queue.cc:237] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "0106efdb5d174dcbb762451807bb5be7" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38735 } }
I20251023 08:00:50.111282 21661 tablet_replica.cc:442] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0106efdb5d174dcbb762451807bb5be7" committed_config { opid_index: -1 peers { permanent_uuid: "0106efdb5d174dcbb762451807bb5be7" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38735 } } }
I20251023 08:00:50.111555 21662 tablet_replica.cc:442] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7: TxnStatusTablet state changed. Reason: New leader 0106efdb5d174dcbb762451807bb5be7. Latest consensus state: current_term: 1 leader_uuid: "0106efdb5d174dcbb762451807bb5be7" committed_config { opid_index: -1 peers { permanent_uuid: "0106efdb5d174dcbb762451807bb5be7" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38735 } } }
I20251023 08:00:50.112001 21661 tablet_replica.cc:445] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7: This TxnStatusTablet replica's current role is: LEADER
I20251023 08:00:50.112082 21662 tablet_replica.cc:445] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7: This TxnStatusTablet replica's current role is: LEADER
W20251023 08:00:50.114094 21513 catalog_manager.cc:5482] Ignoring report from unknown tablet 11111111111111111111111111111111
I20251023 08:00:50.114142 21665 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20251023 08:00:50.114768 21665 txn_status_manager.cc:930] Loading transaction status metadata into memory...
W20251023 08:00:50.119515 21513 catalog_manager.cc:5482] Ignoring report from unknown tablet 11111111111111111111111111111111
I20251023 08:00:50.150265 18906 tablet_server.cc:178] TabletServer@127.18.118.129:0 shutting down...
I20251023 08:00:50.171694 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:50.172981 18906 tablet_replica.cc:333] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7: stopping tablet replica
I20251023 08:00:50.173666 18906 raft_consensus.cc:2243] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 1 LEADER]: Raft consensus shutting down.
I20251023 08:00:50.174299 18906 raft_consensus.cc:2272] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:50.188699 18906 tablet_server.cc:195] TabletServer@127.18.118.129:0 shutdown complete.
I20251023 08:00:50.199481 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:50.205298 21668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:50.205348 21669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:50.208192 18906 server_base.cc:1047] running on GCE node
W20251023 08:00:50.208926 21671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:50.209918 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:50.210196 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:50.210389 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206450210374 us; error 0 us; skew 500 ppm
I20251023 08:00:50.211104 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:50.213922 18906 webserver.cc:492] Webserver started at http://127.18.118.129:42681/ using document root <none> and password file <none>
I20251023 08:00:50.214569 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:50.214787 18906 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251023 08:00:50.219385 18906 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.005s	sys 0.000s
I20251023 08:00:50.222853 21676 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:50.223680 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20251023 08:00:50.223965 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-0-root
uuid: "0106efdb5d174dcbb762451807bb5be7"
format_stamp: "Formatted at 2025-10-23 08:00:49 on dist-test-slave-wkn7"
I20251023 08:00:50.224220 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:50.248520 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:50.249539 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:50.256539 21685 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251023 08:00:50.263535 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251023 08:00:50.263948 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.009s	user 0.002s	sys 0.000s
I20251023 08:00:50.264341 18906 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251023 08:00:50.270777 18906 ts_tablet_manager.cc:616] Registered 1 tablets
I20251023 08:00:50.271126 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.007s	user 0.004s	sys 0.003s
I20251023 08:00:50.272701 21685 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7: Bootstrap starting.
I20251023 08:00:50.309053 21685 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 08:00:50.310106 21685 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7: Bootstrap complete.
I20251023 08:00:50.310770 21685 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7: Time spent bootstrapping tablet: real 0.038s	user 0.022s	sys 0.011s
I20251023 08:00:50.313429 21685 raft_consensus.cc:359] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "0106efdb5d174dcbb762451807bb5be7" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38735 } }
I20251023 08:00:50.314019 21685 raft_consensus.cc:740] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0106efdb5d174dcbb762451807bb5be7, State: Initialized, Role: FOLLOWER
I20251023 08:00:50.314862 21685 consensus_queue.cc:260] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "0106efdb5d174dcbb762451807bb5be7" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38735 } }
I20251023 08:00:50.315536 21685 raft_consensus.cc:399] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 08:00:50.315837 21685 raft_consensus.cc:493] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 08:00:50.316185 21685 raft_consensus.cc:3060] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:50.323199 21685 raft_consensus.cc:515] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "0106efdb5d174dcbb762451807bb5be7" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38735 } }
I20251023 08:00:50.324051 21685 leader_election.cc:304] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0106efdb5d174dcbb762451807bb5be7; no voters: 
I20251023 08:00:50.329703 21685 leader_election.cc:290] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [CANDIDATE]: Term 2 election: Requested vote from peers 
I20251023 08:00:50.330276 21715 raft_consensus.cc:2804] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 08:00:50.338106 21715 raft_consensus.cc:697] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 2 LEADER]: Becoming Leader. State: Replica: 0106efdb5d174dcbb762451807bb5be7, State: Running, Role: LEADER
I20251023 08:00:50.339421 21715 consensus_queue.cc:237] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "0106efdb5d174dcbb762451807bb5be7" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38735 } }
I20251023 08:00:50.348692 21685 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7: Time spent starting tablet: real 0.038s	user 0.017s	sys 0.019s
I20251023 08:00:50.350901 21718 tablet_replica.cc:442] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0106efdb5d174dcbb762451807bb5be7" committed_config { opid_index: -1 peers { permanent_uuid: "0106efdb5d174dcbb762451807bb5be7" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38735 } } }
I20251023 08:00:50.351495 21718 tablet_replica.cc:445] This TxnStatusTablet replica's current role is: LEADER
I20251023 08:00:50.352097 21717 tablet_replica.cc:442] TxnStatusTablet state changed. Reason: New leader 0106efdb5d174dcbb762451807bb5be7. Latest consensus state: current_term: 2 leader_uuid: "0106efdb5d174dcbb762451807bb5be7" committed_config { opid_index: -1 peers { permanent_uuid: "0106efdb5d174dcbb762451807bb5be7" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 38735 } } }
I20251023 08:00:50.352767 21717 tablet_replica.cc:445] This TxnStatusTablet replica's current role is: LEADER
I20251023 08:00:50.364760 21723 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20251023 08:00:50.365494 21723 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20251023 08:00:50.391356 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.129:38735
I20251023 08:00:50.391518 21753 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.129:38735 every 8 connection(s)
I20251023 08:00:50.397650 18906 tablet_server.cc:178] TabletServer@127.18.118.129:38735 shutting down...
I20251023 08:00:50.399708 21754 heartbeater.cc:344] Connected to a master server at 127.18.118.190:35937
I20251023 08:00:50.400732 21754 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:50.401711 21754 heartbeater.cc:507] Master 127.18.118.190:35937 requested a full tablet report, sending...
I20251023 08:00:50.404362 21513 ts_manager.cc:194] Re-registered known tserver with Master: 0106efdb5d174dcbb762451807bb5be7 (127.18.118.129:38735)
W20251023 08:00:50.404908 21513 catalog_manager.cc:5482] Ignoring report from unknown tablet 11111111111111111111111111111111
I20251023 08:00:50.406596 21513 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54150
I20251023 08:00:50.409472 21754 heartbeater.cc:499] Master 127.18.118.190:35937 was elected leader, sending a full tablet report...
W20251023 08:00:50.412338 21513 catalog_manager.cc:5482] Ignoring report from unknown tablet 11111111111111111111111111111111
I20251023 08:00:50.431108 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:50.431936 18906 tablet_replica.cc:333] stopping tablet replica
I20251023 08:00:50.432430 18906 raft_consensus.cc:2243] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 2 LEADER]: Raft consensus shutting down.
I20251023 08:00:50.432857 18906 raft_consensus.cc:2272] T 11111111111111111111111111111111 P 0106efdb5d174dcbb762451807bb5be7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:50.453954 18906 tablet_server.cc:195] TabletServer@127.18.118.129:38735 shutdown complete.
I20251023 08:00:50.467983 18906 master.cc:561] Master@127.18.118.190:35937 shutting down...
I20251023 08:00:50.484877 18906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885 [term 1 LEADER]: Raft consensus shutting down.
I20251023 08:00:50.485442 18906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:50.485738 18906 tablet_replica.cc:333] T 00000000000000000000000000000000 P cdfcc0a709f940c6a271b3232acfe885: stopping tablet replica
I20251023 08:00:50.503960 18906 master.cc:583] Master@127.18.118.190:35937 shutdown complete.
[       OK ] TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet (802 ms)
[ RUN      ] TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet
I20251023 08:00:50.528924 18906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.18.118.190:40163
I20251023 08:00:50.529971 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:50.535149 21760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:50.535380 21761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:50.538712 21763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:50.539114 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:50.539934 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:50.540143 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:50.540309 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206450540290 us; error 0 us; skew 500 ppm
I20251023 08:00:50.540870 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:50.543968 18906 webserver.cc:492] Webserver started at http://127.18.118.190:34789/ using document root <none> and password file <none>
I20251023 08:00:50.544481 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:50.544662 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:50.544932 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:50.546104 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/master-0-root/instance:
uuid: "0e62fd7d3650444891832386a6fd93f3"
format_stamp: "Formatted at 2025-10-23 08:00:50 on dist-test-slave-wkn7"
I20251023 08:00:50.551204 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20251023 08:00:50.554991 21768 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:50.555868 18906 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20251023 08:00:50.556252 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/master-0-root
uuid: "0e62fd7d3650444891832386a6fd93f3"
format_stamp: "Formatted at 2025-10-23 08:00:50 on dist-test-slave-wkn7"
I20251023 08:00:50.556607 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:50.573763 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:50.575119 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:50.621843 21819 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.190:40163 every 8 connection(s)
I20251023 08:00:50.621915 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.190:40163
I20251023 08:00:50.628863 21820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:50.640122 21820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3: Bootstrap starting.
I20251023 08:00:50.644562 21820 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:50.648772 21820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3: No bootstrap required, opened a new log
I20251023 08:00:50.650713 21820 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e62fd7d3650444891832386a6fd93f3" member_type: VOTER }
I20251023 08:00:50.651080 21820 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:50.651310 21820 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e62fd7d3650444891832386a6fd93f3, State: Initialized, Role: FOLLOWER
I20251023 08:00:50.651902 21820 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e62fd7d3650444891832386a6fd93f3" member_type: VOTER }
I20251023 08:00:50.652343 21820 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 08:00:50.652559 21820 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 08:00:50.652804 21820 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:50.657704 21820 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e62fd7d3650444891832386a6fd93f3" member_type: VOTER }
I20251023 08:00:50.658233 21820 leader_election.cc:304] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e62fd7d3650444891832386a6fd93f3; no voters: 
I20251023 08:00:50.659641 21820 leader_election.cc:290] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20251023 08:00:50.659965 21823 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:50.661612 21823 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3 [term 1 LEADER]: Becoming Leader. State: Replica: 0e62fd7d3650444891832386a6fd93f3, State: Running, Role: LEADER
I20251023 08:00:50.662364 21823 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e62fd7d3650444891832386a6fd93f3" member_type: VOTER }
I20251023 08:00:50.663196 21820 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 08:00:50.665701 21824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e62fd7d3650444891832386a6fd93f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e62fd7d3650444891832386a6fd93f3" member_type: VOTER } }
I20251023 08:00:50.665711 21825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e62fd7d3650444891832386a6fd93f3. Latest consensus state: current_term: 1 leader_uuid: "0e62fd7d3650444891832386a6fd93f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e62fd7d3650444891832386a6fd93f3" member_type: VOTER } }
I20251023 08:00:50.666435 21824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:50.666579 21825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:50.679991 21828 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 08:00:50.687548 21828 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 08:00:50.688733 18906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 08:00:50.698668 21828 catalog_manager.cc:1357] Generated new cluster ID: ee75ee8a74d1457893319f9e0e22e6f8
I20251023 08:00:50.698956 21828 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 08:00:50.732220 21828 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 08:00:50.733912 21828 catalog_manager.cc:1514] Loading token signing keys...
I20251023 08:00:50.750108 21828 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3: Generated new TSK 0
I20251023 08:00:50.750731 21828 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 08:00:50.755791 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:50.762765 21841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:50.764765 21842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:50.766815 18906 server_base.cc:1047] running on GCE node
W20251023 08:00:50.767274 21844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:50.768314 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:50.768519 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:50.768692 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206450768671 us; error 0 us; skew 500 ppm
I20251023 08:00:50.769248 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:50.772131 18906 webserver.cc:492] Webserver started at http://127.18.118.129:33635/ using document root <none> and password file <none>
I20251023 08:00:50.772619 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:50.772801 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:50.773053 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:50.774219 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-0-root/instance:
uuid: "757edec740154ae6bc20d4a67971ab15"
format_stamp: "Formatted at 2025-10-23 08:00:50 on dist-test-slave-wkn7"
I20251023 08:00:50.779067 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20251023 08:00:50.782804 21849 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:50.783645 18906 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20251023 08:00:50.783916 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-0-root
uuid: "757edec740154ae6bc20d4a67971ab15"
format_stamp: "Formatted at 2025-10-23 08:00:50 on dist-test-slave-wkn7"
I20251023 08:00:50.784173 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:50.800940 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:50.801985 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:50.814296 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:50.814711 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:50.815090 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:50.815351 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:50.883894 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.129:34173
I20251023 08:00:50.884007 21919 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.129:34173 every 8 connection(s)
I20251023 08:00:50.889755 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251023 08:00:50.890002 21920 heartbeater.cc:344] Connected to a master server at 127.18.118.190:40163
I20251023 08:00:50.890440 21920 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:50.891342 21920 heartbeater.cc:507] Master 127.18.118.190:40163 requested a full tablet report, sending...
I20251023 08:00:50.893738 21785 ts_manager.cc:194] Registered new tserver with Master: 757edec740154ae6bc20d4a67971ab15 (127.18.118.129:34173)
W20251023 08:00:50.897491 21922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:50.897531 21923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:50.899019 21785 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55946
W20251023 08:00:50.905028 21925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:50.906949 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:50.907699 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:50.907907 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:50.908066 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206450908047 us; error 0 us; skew 500 ppm
I20251023 08:00:50.908530 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:50.911351 18906 webserver.cc:492] Webserver started at http://127.18.118.130:41601/ using document root <none> and password file <none>
I20251023 08:00:50.911813 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:50.911998 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:50.912252 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:50.913326 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-1-root/instance:
uuid: "d3f4e7f2ef9c453183af3f0b585c3c05"
format_stamp: "Formatted at 2025-10-23 08:00:50 on dist-test-slave-wkn7"
I20251023 08:00:50.918470 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20251023 08:00:50.921847 21930 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:50.922683 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.002s
I20251023 08:00:50.922947 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-1-root
uuid: "d3f4e7f2ef9c453183af3f0b585c3c05"
format_stamp: "Formatted at 2025-10-23 08:00:50 on dist-test-slave-wkn7"
I20251023 08:00:50.923206 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:50.948765 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:50.949819 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:50.956398 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:50.956730 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:50.957027 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:50.957293 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20251023 08:00:51.025431 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.130:38483
I20251023 08:00:51.025611 21999 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.130:38483 every 8 connection(s)
I20251023 08:00:51.030455 22000 heartbeater.cc:344] Connected to a master server at 127.18.118.190:40163
I20251023 08:00:51.030822 22000 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:51.031576 22000 heartbeater.cc:507] Master 127.18.118.190:40163 requested a full tablet report, sending...
I20251023 08:00:51.033860 21785 ts_manager.cc:194] Registered new tserver with Master: d3f4e7f2ef9c453183af3f0b585c3c05 (127.18.118.130:38483)
I20251023 08:00:51.034015 18906 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.005275149s
I20251023 08:00:51.035727 21785 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55974
I20251023 08:00:51.068418 21885 tablet_service.cc:1505] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20251023 08:00:51.070015 21885 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:51.086901 22010 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15: Bootstrap starting.
I20251023 08:00:51.092896 22010 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:51.097965 22010 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15: No bootstrap required, opened a new log
I20251023 08:00:51.098450 22010 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15: Time spent bootstrapping tablet: real 0.012s	user 0.005s	sys 0.005s
I20251023 08:00:51.101038 22010 raft_consensus.cc:359] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "757edec740154ae6bc20d4a67971ab15" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34173 } }
I20251023 08:00:51.101650 22010 raft_consensus.cc:385] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:51.101934 22010 raft_consensus.cc:740] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 757edec740154ae6bc20d4a67971ab15, State: Initialized, Role: FOLLOWER
I20251023 08:00:51.102612 22010 consensus_queue.cc:260] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "757edec740154ae6bc20d4a67971ab15" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34173 } }
I20251023 08:00:51.103252 22010 raft_consensus.cc:399] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 08:00:51.103545 22010 raft_consensus.cc:493] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 08:00:51.103876 22010 raft_consensus.cc:3060] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:51.110641 22010 raft_consensus.cc:515] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "757edec740154ae6bc20d4a67971ab15" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34173 } }
I20251023 08:00:51.111361 22010 leader_election.cc:304] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 757edec740154ae6bc20d4a67971ab15; no voters: 
I20251023 08:00:51.112831 22010 leader_election.cc:290] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20251023 08:00:51.113152 22012 raft_consensus.cc:2804] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:51.115056 21920 heartbeater.cc:499] Master 127.18.118.190:40163 was elected leader, sending a full tablet report...
I20251023 08:00:51.115453 22010 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15: Time spent starting tablet: real 0.017s	user 0.016s	sys 0.001s
I20251023 08:00:51.116462 22012 raft_consensus.cc:697] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 1 LEADER]: Becoming Leader. State: Replica: 757edec740154ae6bc20d4a67971ab15, State: Running, Role: LEADER
I20251023 08:00:51.117089 22012 consensus_queue.cc:237] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "757edec740154ae6bc20d4a67971ab15" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34173 } }
I20251023 08:00:51.122450 22013 tablet_replica.cc:442] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "757edec740154ae6bc20d4a67971ab15" committed_config { opid_index: -1 peers { permanent_uuid: "757edec740154ae6bc20d4a67971ab15" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34173 } } }
I20251023 08:00:51.123015 22013 tablet_replica.cc:445] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15: This TxnStatusTablet replica's current role is: LEADER
I20251023 08:00:51.124338 22014 tablet_replica.cc:442] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15: TxnStatusTablet state changed. Reason: New leader 757edec740154ae6bc20d4a67971ab15. Latest consensus state: current_term: 1 leader_uuid: "757edec740154ae6bc20d4a67971ab15" committed_config { opid_index: -1 peers { permanent_uuid: "757edec740154ae6bc20d4a67971ab15" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34173 } } }
I20251023 08:00:51.125196 22014 tablet_replica.cc:445] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15: This TxnStatusTablet replica's current role is: LEADER
W20251023 08:00:51.125319 21785 catalog_manager.cc:5482] Ignoring report from unknown tablet 11111111111111111111111111111111
I20251023 08:00:51.130007 22017 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
W20251023 08:00:51.130412 21785 catalog_manager.cc:5482] Ignoring report from unknown tablet 11111111111111111111111111111111
I20251023 08:00:51.130481 22017 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20251023 08:00:51.189740 22021 ts_tablet_manager.cc:933] T 11111111111111111111111111111111 P d3f4e7f2ef9c453183af3f0b585c3c05: Initiating tablet copy from peer 757edec740154ae6bc20d4a67971ab15 (127.18.118.129:34173)
I20251023 08:00:51.191351 22021 tablet_copy_client.cc:323] T 11111111111111111111111111111111 P d3f4e7f2ef9c453183af3f0b585c3c05: tablet copy: Beginning tablet copy session from remote peer at address 127.18.118.129:34173
I20251023 08:00:51.200695 21905 tablet_copy_service.cc:140] P 757edec740154ae6bc20d4a67971ab15: Received BeginTabletCopySession request for tablet 11111111111111111111111111111111 from peer d3f4e7f2ef9c453183af3f0b585c3c05 ({username='slave'} at 127.0.0.1:41794)
I20251023 08:00:51.201208 21905 tablet_copy_service.cc:161] P 757edec740154ae6bc20d4a67971ab15: Beginning new tablet copy session on tablet 11111111111111111111111111111111 from peer d3f4e7f2ef9c453183af3f0b585c3c05 at {username='slave'} at 127.0.0.1:41794: session id = d3f4e7f2ef9c453183af3f0b585c3c05-11111111111111111111111111111111
I20251023 08:00:51.206686 21905 tablet_copy_source_session.cc:215] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 08:00:51.210040 22021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:51.225335 22021 tablet_copy_client.cc:806] T 11111111111111111111111111111111 P d3f4e7f2ef9c453183af3f0b585c3c05: tablet copy: Starting download of 0 data blocks...
I20251023 08:00:51.225904 22021 tablet_copy_client.cc:670] T 11111111111111111111111111111111 P d3f4e7f2ef9c453183af3f0b585c3c05: tablet copy: Starting download of 1 WAL segments...
I20251023 08:00:51.229081 22021 tablet_copy_client.cc:538] T 11111111111111111111111111111111 P d3f4e7f2ef9c453183af3f0b585c3c05: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 08:00:51.238226 22021 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P d3f4e7f2ef9c453183af3f0b585c3c05: Bootstrap starting.
I20251023 08:00:51.267091 22021 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P d3f4e7f2ef9c453183af3f0b585c3c05: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 08:00:51.268008 22021 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P d3f4e7f2ef9c453183af3f0b585c3c05: Bootstrap complete.
I20251023 08:00:51.268685 22021 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P d3f4e7f2ef9c453183af3f0b585c3c05: Time spent bootstrapping tablet: real 0.031s	user 0.027s	sys 0.000s
I20251023 08:00:51.271198 22021 raft_consensus.cc:359] T 11111111111111111111111111111111 P d3f4e7f2ef9c453183af3f0b585c3c05 [term 1 NON_PARTICIPANT]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "757edec740154ae6bc20d4a67971ab15" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34173 } }
I20251023 08:00:51.271741 22021 raft_consensus.cc:740] T 11111111111111111111111111111111 P d3f4e7f2ef9c453183af3f0b585c3c05 [term 1 NON_PARTICIPANT]: Becoming Follower/Learner. State: Replica: d3f4e7f2ef9c453183af3f0b585c3c05, State: Initialized, Role: NON_PARTICIPANT
I20251023 08:00:51.272341 22021 consensus_queue.cc:260] T 11111111111111111111111111111111 P d3f4e7f2ef9c453183af3f0b585c3c05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "757edec740154ae6bc20d4a67971ab15" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34173 } }
I20251023 08:00:51.274025 22024 tablet_replica.cc:442] T 11111111111111111111111111111111 P d3f4e7f2ef9c453183af3f0b585c3c05: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 peers { permanent_uuid: "757edec740154ae6bc20d4a67971ab15" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34173 } } }
I20251023 08:00:51.274184 22000 heartbeater.cc:499] Master 127.18.118.190:40163 was elected leader, sending a full tablet report...
I20251023 08:00:51.274575 22024 tablet_replica.cc:445] T 11111111111111111111111111111111 P d3f4e7f2ef9c453183af3f0b585c3c05: This TxnStatusTablet replica's current role is: NON_PARTICIPANT
I20251023 08:00:51.274763 22021 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P d3f4e7f2ef9c453183af3f0b585c3c05: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
W20251023 08:00:51.276892 21785 catalog_manager.cc:5482] Ignoring report from unknown tablet 11111111111111111111111111111111
I20251023 08:00:51.277343 21905 tablet_copy_service.cc:342] P 757edec740154ae6bc20d4a67971ab15: Request end of tablet copy session d3f4e7f2ef9c453183af3f0b585c3c05-11111111111111111111111111111111 received from {username='slave'} at 127.0.0.1:41794
I20251023 08:00:51.277736 21905 tablet_copy_service.cc:434] P 757edec740154ae6bc20d4a67971ab15: ending tablet copy session d3f4e7f2ef9c453183af3f0b585c3c05-11111111111111111111111111111111 on tablet 11111111111111111111111111111111 with peer d3f4e7f2ef9c453183af3f0b585c3c05
W20251023 08:00:51.280742 21785 catalog_manager.cc:5482] Ignoring report from unknown tablet 11111111111111111111111111111111
I20251023 08:00:51.290485 18906 tablet_server.cc:178] TabletServer@127.18.118.129:0 shutting down...
I20251023 08:00:51.311159 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:51.312000 18906 tablet_replica.cc:333] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15: stopping tablet replica
I20251023 08:00:51.312548 18906 raft_consensus.cc:2243] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 1 LEADER]: Raft consensus shutting down.
I20251023 08:00:51.313019 18906 raft_consensus.cc:2272] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:51.324615 18906 tablet_server.cc:195] TabletServer@127.18.118.129:0 shutdown complete.
I20251023 08:00:51.336771 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:51.343199 22025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:51.343232 22026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:51.346311 22028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:51.346796 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:51.347826 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:51.348071 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:51.348261 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206451348242 us; error 0 us; skew 500 ppm
I20251023 08:00:51.348909 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:51.352566 18906 webserver.cc:492] Webserver started at http://127.18.118.129:33635/ using document root <none> and password file <none>
I20251023 08:00:51.353241 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:51.353475 18906 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251023 08:00:51.357810 18906 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20251023 08:00:51.360893 22033 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:51.361776 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20251023 08:00:51.362123 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-0-root
uuid: "757edec740154ae6bc20d4a67971ab15"
format_stamp: "Formatted at 2025-10-23 08:00:50 on dist-test-slave-wkn7"
I20251023 08:00:51.362515 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1761206396810293-18906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:51.379060 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:51.380453 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:51.388554 22042 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251023 08:00:51.396268 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251023 08:00:51.396536 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.009s	user 0.001s	sys 0.000s
I20251023 08:00:51.396884 18906 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251023 08:00:51.404186 18906 ts_tablet_manager.cc:616] Registered 1 tablets
I20251023 08:00:51.404474 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.008s	user 0.005s	sys 0.000s
I20251023 08:00:51.404531 22042 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15: Bootstrap starting.
I20251023 08:00:51.440261 22042 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 08:00:51.441308 22042 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15: Bootstrap complete.
I20251023 08:00:51.441960 22042 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15: Time spent bootstrapping tablet: real 0.038s	user 0.026s	sys 0.010s
I20251023 08:00:51.444742 22042 raft_consensus.cc:359] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "757edec740154ae6bc20d4a67971ab15" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34173 } }
I20251023 08:00:51.445341 22042 raft_consensus.cc:740] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 757edec740154ae6bc20d4a67971ab15, State: Initialized, Role: FOLLOWER
I20251023 08:00:51.446170 22042 consensus_queue.cc:260] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "757edec740154ae6bc20d4a67971ab15" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34173 } }
I20251023 08:00:51.446946 22042 raft_consensus.cc:399] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 08:00:51.447249 22042 raft_consensus.cc:493] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 08:00:51.447597 22042 raft_consensus.cc:3060] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:51.454813 22042 raft_consensus.cc:515] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "757edec740154ae6bc20d4a67971ab15" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34173 } }
I20251023 08:00:51.455524 22042 leader_election.cc:304] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 757edec740154ae6bc20d4a67971ab15; no voters: 
I20251023 08:00:51.457643 22042 leader_election.cc:290] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [CANDIDATE]: Term 2 election: Requested vote from peers 
I20251023 08:00:51.458173 22061 raft_consensus.cc:2804] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 08:00:51.463150 22061 raft_consensus.cc:697] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 2 LEADER]: Becoming Leader. State: Replica: 757edec740154ae6bc20d4a67971ab15, State: Running, Role: LEADER
I20251023 08:00:51.464605 22061 consensus_queue.cc:237] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "757edec740154ae6bc20d4a67971ab15" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34173 } }
I20251023 08:00:51.467948 22042 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15: Time spent starting tablet: real 0.026s	user 0.014s	sys 0.008s
I20251023 08:00:51.471345 22063 tablet_replica.cc:442] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "757edec740154ae6bc20d4a67971ab15" committed_config { opid_index: -1 peers { permanent_uuid: "757edec740154ae6bc20d4a67971ab15" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34173 } } }
I20251023 08:00:51.472045 22063 tablet_replica.cc:445] This TxnStatusTablet replica's current role is: LEADER
I20251023 08:00:51.471721 22062 tablet_replica.cc:442] TxnStatusTablet state changed. Reason: New leader 757edec740154ae6bc20d4a67971ab15. Latest consensus state: current_term: 2 leader_uuid: "757edec740154ae6bc20d4a67971ab15" committed_config { opid_index: -1 peers { permanent_uuid: "757edec740154ae6bc20d4a67971ab15" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 34173 } } }
I20251023 08:00:51.472769 22062 tablet_replica.cc:445] This TxnStatusTablet replica's current role is: LEADER
I20251023 08:00:51.484261 22067 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20251023 08:00:51.484854 22067 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20251023 08:00:51.518980 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.129:34173
I20251023 08:00:51.519070 22110 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.129:34173 every 8 connection(s)
I20251023 08:00:51.525339 22111 heartbeater.cc:344] Connected to a master server at 127.18.118.190:40163
I20251023 08:00:51.525802 22111 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:51.526955 18906 tablet_server.cc:178] TabletServer@127.18.118.129:34173 shutting down...
I20251023 08:00:51.530936 22111 heartbeater.cc:507] Master 127.18.118.190:40163 requested a full tablet report, sending...
I20251023 08:00:51.533811 21785 ts_manager.cc:194] Re-registered known tserver with Master: 757edec740154ae6bc20d4a67971ab15 (127.18.118.129:34173)
W20251023 08:00:51.534309 21785 catalog_manager.cc:5482] Ignoring report from unknown tablet 11111111111111111111111111111111
I20251023 08:00:51.536041 21785 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56012
I20251023 08:00:51.540601 22111 heartbeater.cc:499] Master 127.18.118.190:40163 was elected leader, sending a full tablet report...
W20251023 08:00:51.545681 21785 catalog_manager.cc:5482] Ignoring report from unknown tablet 11111111111111111111111111111111
I20251023 08:00:51.559051 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:51.559875 18906 tablet_replica.cc:333] stopping tablet replica
I20251023 08:00:51.560429 18906 raft_consensus.cc:2243] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 2 LEADER]: Raft consensus shutting down.
I20251023 08:00:51.560870 18906 raft_consensus.cc:2272] T 11111111111111111111111111111111 P 757edec740154ae6bc20d4a67971ab15 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:51.573905 18906 tablet_server.cc:195] TabletServer@127.18.118.129:34173 shutdown complete.
I20251023 08:00:51.584493 18906 tablet_server.cc:178] TabletServer@127.18.118.130:0 shutting down...
I20251023 08:00:51.613711 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:51.614557 18906 tablet_replica.cc:333] T 11111111111111111111111111111111 P d3f4e7f2ef9c453183af3f0b585c3c05: stopping tablet replica
I20251023 08:00:51.615028 18906 raft_consensus.cc:2243] T 11111111111111111111111111111111 P d3f4e7f2ef9c453183af3f0b585c3c05 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20251023 08:00:51.615372 18906 raft_consensus.cc:2272] T 11111111111111111111111111111111 P d3f4e7f2ef9c453183af3f0b585c3c05 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20251023 08:00:51.636067 18906 tablet_server.cc:195] TabletServer@127.18.118.130:0 shutdown complete.
I20251023 08:00:51.649696 18906 master.cc:561] Master@127.18.118.190:40163 shutting down...
I20251023 08:00:51.668058 18906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3 [term 1 LEADER]: Raft consensus shutting down.
I20251023 08:00:51.668668 18906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:51.669045 18906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0e62fd7d3650444891832386a6fd93f3: stopping tablet replica
I20251023 08:00:51.688742 18906 master.cc:583] Master@127.18.118.190:40163 shutdown complete.
[       OK ] TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet (1186 ms)
[ RUN      ] TxnStatusTabletManagementTest.TestTabletServerProxyCalls
I20251023 08:00:51.715693 18906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.18.118.190:36933
I20251023 08:00:51.716863 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:51.722236 22117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:51.722512 22118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:51.725843 22120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:51.725979 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:51.727025 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:51.727280 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:51.727466 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206451727448 us; error 0 us; skew 500 ppm
I20251023 08:00:51.728194 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:51.731436 18906 webserver.cc:492] Webserver started at http://127.18.118.190:39239/ using document root <none> and password file <none>
I20251023 08:00:51.732079 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:51.732306 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:51.732649 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:51.734143 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1761206396810293-18906-0/minicluster-data/master-0-root/instance:
uuid: "f4ba496220884312ac3ceb0f97ceff20"
format_stamp: "Formatted at 2025-10-23 08:00:51 on dist-test-slave-wkn7"
I20251023 08:00:51.740110 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20251023 08:00:51.745342 22125 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:51.746675 18906 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20251023 08:00:51.747099 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1761206396810293-18906-0/minicluster-data/master-0-root
uuid: "f4ba496220884312ac3ceb0f97ceff20"
format_stamp: "Formatted at 2025-10-23 08:00:51 on dist-test-slave-wkn7"
I20251023 08:00:51.747478 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1761206396810293-18906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1761206396810293-18906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1761206396810293-18906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:51.779546 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:51.781381 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:51.834676 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.190:36933
I20251023 08:00:51.834786 22176 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.190:36933 every 8 connection(s)
I20251023 08:00:51.838881 22177 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:51.853785 22177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20: Bootstrap starting.
I20251023 08:00:51.859962 22177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:51.865212 22177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20: No bootstrap required, opened a new log
I20251023 08:00:51.867875 22177 raft_consensus.cc:359] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ba496220884312ac3ceb0f97ceff20" member_type: VOTER }
I20251023 08:00:51.868443 22177 raft_consensus.cc:385] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:51.869065 22177 raft_consensus.cc:740] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4ba496220884312ac3ceb0f97ceff20, State: Initialized, Role: FOLLOWER
I20251023 08:00:51.870043 22177 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ba496220884312ac3ceb0f97ceff20" member_type: VOTER }
I20251023 08:00:51.870766 22177 raft_consensus.cc:399] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 08:00:51.871069 22177 raft_consensus.cc:493] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 08:00:51.871407 22177 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:51.878185 22177 raft_consensus.cc:515] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ba496220884312ac3ceb0f97ceff20" member_type: VOTER }
I20251023 08:00:51.878911 22177 leader_election.cc:304] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4ba496220884312ac3ceb0f97ceff20; no voters: 
I20251023 08:00:51.880260 22177 leader_election.cc:290] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20251023 08:00:51.880657 22180 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:51.882683 22180 raft_consensus.cc:697] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20 [term 1 LEADER]: Becoming Leader. State: Replica: f4ba496220884312ac3ceb0f97ceff20, State: Running, Role: LEADER
I20251023 08:00:51.883383 22180 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ba496220884312ac3ceb0f97ceff20" member_type: VOTER }
I20251023 08:00:51.883859 22177 sys_catalog.cc:565] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 08:00:51.886160 22181 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4ba496220884312ac3ceb0f97ceff20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ba496220884312ac3ceb0f97ceff20" member_type: VOTER } }
I20251023 08:00:51.886879 22181 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:51.886204 22182 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4ba496220884312ac3ceb0f97ceff20. Latest consensus state: current_term: 1 leader_uuid: "f4ba496220884312ac3ceb0f97ceff20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ba496220884312ac3ceb0f97ceff20" member_type: VOTER } }
I20251023 08:00:51.888546 22182 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:51.893760 22185 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 08:00:51.899309 22185 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 08:00:51.902676 18906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 08:00:51.909224 22185 catalog_manager.cc:1357] Generated new cluster ID: c47ab1f482a14c56bf5c99e5b1a0e00b
I20251023 08:00:51.909536 22185 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 08:00:51.921000 22185 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 08:00:51.922313 22185 catalog_manager.cc:1514] Loading token signing keys...
I20251023 08:00:51.937005 22185 catalog_manager.cc:6022] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20: Generated new TSK 0
I20251023 08:00:51.937654 22185 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 08:00:51.969154 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:51.975525 22198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:51.975726 22199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:51.979099 22201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:51.979606 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:51.980647 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:51.980926 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:51.981385 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206451981365 us; error 0 us; skew 500 ppm
I20251023 08:00:51.982194 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:51.986519 18906 webserver.cc:492] Webserver started at http://127.18.118.129:34081/ using document root <none> and password file <none>
I20251023 08:00:51.987360 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:51.987604 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:51.987926 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:51.989763 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1761206396810293-18906-0/minicluster-data/ts-0-root/instance:
uuid: "6efe309ccd7a402eb0622c8a02a3f760"
format_stamp: "Formatted at 2025-10-23 08:00:51 on dist-test-slave-wkn7"
I20251023 08:00:51.996186 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20251023 08:00:52.001549 22206 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:52.002425 18906 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.004s
I20251023 08:00:52.002763 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1761206396810293-18906-0/minicluster-data/ts-0-root
uuid: "6efe309ccd7a402eb0622c8a02a3f760"
format_stamp: "Formatted at 2025-10-23 08:00:51 on dist-test-slave-wkn7"
I20251023 08:00:52.003165 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1761206396810293-18906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1761206396810293-18906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1761206396810293-18906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:52.016662 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:52.018107 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:52.032255 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:52.032568 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:52.032881 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:52.033089 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:52.093147 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.129:41831
I20251023 08:00:52.093346 22276 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.129:41831 every 8 connection(s)
I20251023 08:00:52.097990 22277 heartbeater.cc:344] Connected to a master server at 127.18.118.190:36933
I20251023 08:00:52.098333 22277 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:52.098999 22277 heartbeater.cc:507] Master 127.18.118.190:36933 requested a full tablet report, sending...
I20251023 08:00:52.100757 22142 ts_manager.cc:194] Registered new tserver with Master: 6efe309ccd7a402eb0622c8a02a3f760 (127.18.118.129:41831)
I20251023 08:00:52.101346 18906 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.004856848s
I20251023 08:00:52.102409 22142 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34748
I20251023 08:00:52.136319 22242 tablet_service.cc:1505] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20251023 08:00:52.137825 22242 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:52.153718 22287 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760: Bootstrap starting.
I20251023 08:00:52.158012 22287 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:52.162045 22287 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760: No bootstrap required, opened a new log
I20251023 08:00:52.162401 22287 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760: Time spent bootstrapping tablet: real 0.009s	user 0.004s	sys 0.003s
I20251023 08:00:52.164260 22287 raft_consensus.cc:359] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "6efe309ccd7a402eb0622c8a02a3f760" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41831 } }
I20251023 08:00:52.164664 22287 raft_consensus.cc:385] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:52.164891 22287 raft_consensus.cc:740] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6efe309ccd7a402eb0622c8a02a3f760, State: Initialized, Role: FOLLOWER
I20251023 08:00:52.165565 22287 consensus_queue.cc:260] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "6efe309ccd7a402eb0622c8a02a3f760" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41831 } }
I20251023 08:00:52.166005 22287 raft_consensus.cc:399] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 08:00:52.166347 22287 raft_consensus.cc:493] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 08:00:52.166632 22287 raft_consensus.cc:3060] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:52.171211 22287 raft_consensus.cc:515] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "6efe309ccd7a402eb0622c8a02a3f760" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41831 } }
I20251023 08:00:52.171712 22287 leader_election.cc:304] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6efe309ccd7a402eb0622c8a02a3f760; no voters: 
I20251023 08:00:52.172948 22287 leader_election.cc:290] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20251023 08:00:52.173388 22289 raft_consensus.cc:2804] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:52.174865 22277 heartbeater.cc:499] Master 127.18.118.190:36933 was elected leader, sending a full tablet report...
I20251023 08:00:52.175486 22287 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760: Time spent starting tablet: real 0.013s	user 0.013s	sys 0.000s
I20251023 08:00:52.176121 22289 raft_consensus.cc:697] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760 [term 1 LEADER]: Becoming Leader. State: Replica: 6efe309ccd7a402eb0622c8a02a3f760, State: Running, Role: LEADER
I20251023 08:00:52.176851 22289 consensus_queue.cc:237] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "6efe309ccd7a402eb0622c8a02a3f760" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41831 } }
I20251023 08:00:52.180371 22290 tablet_replica.cc:442] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6efe309ccd7a402eb0622c8a02a3f760" committed_config { opid_index: -1 peers { permanent_uuid: "6efe309ccd7a402eb0622c8a02a3f760" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41831 } } }
I20251023 08:00:52.180874 22290 tablet_replica.cc:445] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760: This TxnStatusTablet replica's current role is: LEADER
I20251023 08:00:52.181938 22291 tablet_replica.cc:442] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760: TxnStatusTablet state changed. Reason: New leader 6efe309ccd7a402eb0622c8a02a3f760. Latest consensus state: current_term: 1 leader_uuid: "6efe309ccd7a402eb0622c8a02a3f760" committed_config { opid_index: -1 peers { permanent_uuid: "6efe309ccd7a402eb0622c8a02a3f760" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41831 } } }
I20251023 08:00:52.182718 22291 tablet_replica.cc:445] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760: This TxnStatusTablet replica's current role is: LEADER
W20251023 08:00:52.183887 22142 catalog_manager.cc:5482] Ignoring report from unknown tablet 11111111111111111111111111111111
I20251023 08:00:52.187270 22293 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20251023 08:00:52.187661 22293 txn_status_manager.cc:930] Loading transaction status metadata into memory...
W20251023 08:00:52.188752 22142 catalog_manager.cc:5482] Ignoring report from unknown tablet 11111111111111111111111111111111
W20251023 08:00:52.241199 22221 meta_cache.cc:788] Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20251023 08:00:52.241554 22221 txn_status_manager.cc:244] Participant participant of txn 1 returned error for BEGIN_COMMIT op, aborting: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20251023 08:00:52.241914 22221 txn_status_manager.cc:206] Scheduling write for ABORT_IN_PROGRESS for txn 1
I20251023 08:00:52.250635 22221 txn_status_manager.cc:337] Participant participant was not found for ABORT_TXN, proceeding as if op succeeded: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20251023 08:00:52.289291 22221 txn_status_manager.cc:244] Participant participant of txn 2 returned error for BEGIN_COMMIT op, aborting: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20251023 08:00:52.289670 22221 txn_status_manager.cc:206] Scheduling write for ABORT_IN_PROGRESS for txn 2
I20251023 08:00:52.296482 22221 txn_status_manager.cc:337] Participant participant was not found for ABORT_TXN, proceeding as if op succeeded: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
W20251023 08:00:52.308432 22242 server_base.cc:1129] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:60394
W20251023 08:00:52.310506 22242 server_base.cc:1129] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:60394
W20251023 08:00:52.312368 22242 server_base.cc:1129] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:60394
W20251023 08:00:52.314262 22242 server_base.cc:1129] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:60394
W20251023 08:00:52.316148 22242 server_base.cc:1129] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:60394
W20251023 08:00:52.318197 22242 server_base.cc:1129] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:60394
W20251023 08:00:52.328734 22242 server_base.cc:1129] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:60410
W20251023 08:00:52.330713 22242 server_base.cc:1129] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:60410
W20251023 08:00:52.332535 22242 server_base.cc:1129] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:60410
W20251023 08:00:52.334386 22242 server_base.cc:1129] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:60410
W20251023 08:00:52.336117 22242 server_base.cc:1129] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:60410
W20251023 08:00:52.337880 22242 server_base.cc:1129] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:60410
I20251023 08:00:52.340548 18906 tablet_server.cc:178] TabletServer@127.18.118.129:0 shutting down...
I20251023 08:00:52.372547 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:52.373680 18906 tablet_replica.cc:333] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760: stopping tablet replica
I20251023 08:00:52.374166 18906 raft_consensus.cc:2243] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760 [term 1 LEADER]: Raft consensus shutting down.
I20251023 08:00:52.374646 18906 raft_consensus.cc:2272] T 11111111111111111111111111111111 P 6efe309ccd7a402eb0622c8a02a3f760 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:52.386451 18906 tablet_server.cc:195] TabletServer@127.18.118.129:0 shutdown complete.
I20251023 08:00:52.396533 18906 master.cc:561] Master@127.18.118.190:36933 shutting down...
I20251023 08:00:52.419607 18906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20 [term 1 LEADER]: Raft consensus shutting down.
I20251023 08:00:52.420210 18906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:52.420686 18906 tablet_replica.cc:333] T 00000000000000000000000000000000 P f4ba496220884312ac3ceb0f97ceff20: stopping tablet replica
I20251023 08:00:52.431860 18906 master.cc:583] Master@127.18.118.190:36933 shutdown complete.
[       OK ] TxnStatusTabletManagementTest.TestTabletServerProxyCalls (744 ms)
[ RUN      ] TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors
I20251023 08:00:52.461851 18906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.18.118.190:46545
I20251023 08:00:52.462971 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:52.468842 22302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:52.469738 22303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:52.472393 18906 server_base.cc:1047] running on GCE node
W20251023 08:00:52.473232 22306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:52.474229 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:52.474475 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:52.474660 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206452474629 us; error 0 us; skew 500 ppm
I20251023 08:00:52.475286 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:52.478081 18906 webserver.cc:492] Webserver started at http://127.18.118.190:44937/ using document root <none> and password file <none>
I20251023 08:00:52.478714 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:52.478935 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:52.479218 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:52.480576 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1761206396810293-18906-0/minicluster-data/master-0-root/instance:
uuid: "15747058c6f749ea8f7761f221bf73c3"
format_stamp: "Formatted at 2025-10-23 08:00:52 on dist-test-slave-wkn7"
I20251023 08:00:52.486352 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20251023 08:00:52.490898 22310 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:52.491832 18906 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20251023 08:00:52.492179 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1761206396810293-18906-0/minicluster-data/master-0-root
uuid: "15747058c6f749ea8f7761f221bf73c3"
format_stamp: "Formatted at 2025-10-23 08:00:52 on dist-test-slave-wkn7"
I20251023 08:00:52.492551 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1761206396810293-18906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1761206396810293-18906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1761206396810293-18906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:52.512791 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:52.514160 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:52.560065 22361 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.190:46545 every 8 connection(s)
I20251023 08:00:52.560067 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.190:46545
I20251023 08:00:52.565431 22362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:52.575552 22362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3: Bootstrap starting.
I20251023 08:00:52.580010 22362 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:52.584645 22362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3: No bootstrap required, opened a new log
I20251023 08:00:52.587085 22362 raft_consensus.cc:359] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15747058c6f749ea8f7761f221bf73c3" member_type: VOTER }
I20251023 08:00:52.587617 22362 raft_consensus.cc:385] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:52.587898 22362 raft_consensus.cc:740] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15747058c6f749ea8f7761f221bf73c3, State: Initialized, Role: FOLLOWER
I20251023 08:00:52.588617 22362 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15747058c6f749ea8f7761f221bf73c3" member_type: VOTER }
I20251023 08:00:52.589241 22362 raft_consensus.cc:399] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 08:00:52.589553 22362 raft_consensus.cc:493] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 08:00:52.589882 22362 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:52.596225 22362 raft_consensus.cc:515] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15747058c6f749ea8f7761f221bf73c3" member_type: VOTER }
I20251023 08:00:52.596719 22362 leader_election.cc:304] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15747058c6f749ea8f7761f221bf73c3; no voters: 
I20251023 08:00:52.598311 22362 leader_election.cc:290] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20251023 08:00:52.598781 22365 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:52.601157 22365 raft_consensus.cc:697] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3 [term 1 LEADER]: Becoming Leader. State: Replica: 15747058c6f749ea8f7761f221bf73c3, State: Running, Role: LEADER
I20251023 08:00:52.601920 22365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15747058c6f749ea8f7761f221bf73c3" member_type: VOTER }
I20251023 08:00:52.602308 22362 sys_catalog.cc:565] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 08:00:52.605852 22366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15747058c6f749ea8f7761f221bf73c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15747058c6f749ea8f7761f221bf73c3" member_type: VOTER } }
I20251023 08:00:52.606354 22366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:52.607632 22367 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15747058c6f749ea8f7761f221bf73c3. Latest consensus state: current_term: 1 leader_uuid: "15747058c6f749ea8f7761f221bf73c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15747058c6f749ea8f7761f221bf73c3" member_type: VOTER } }
I20251023 08:00:52.608125 22367 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:52.612610 22370 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 08:00:52.619131 22370 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 08:00:52.622606 18906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 08:00:52.629323 22370 catalog_manager.cc:1357] Generated new cluster ID: 63231603e8864ab7bb34e9422ef5d43a
I20251023 08:00:52.629617 22370 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 08:00:52.652595 22370 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 08:00:52.653825 22370 catalog_manager.cc:1514] Loading token signing keys...
I20251023 08:00:52.668644 22370 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3: Generated new TSK 0
I20251023 08:00:52.669235 22370 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 08:00:52.690893 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:52.696532 22383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:52.697831 22384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:52.699281 22386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:52.699522 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:52.700284 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:52.700465 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:52.700587 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206452700577 us; error 0 us; skew 500 ppm
I20251023 08:00:52.701027 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:52.703341 18906 webserver.cc:492] Webserver started at http://127.18.118.129:34417/ using document root <none> and password file <none>
I20251023 08:00:52.703884 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:52.704046 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:52.704254 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:52.705330 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1761206396810293-18906-0/minicluster-data/ts-0-root/instance:
uuid: "40126b833a4642c7b8567ae1c9c717ea"
format_stamp: "Formatted at 2025-10-23 08:00:52 on dist-test-slave-wkn7"
I20251023 08:00:52.709578 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20251023 08:00:52.712659 22391 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:52.713425 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20251023 08:00:52.713687 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1761206396810293-18906-0/minicluster-data/ts-0-root
uuid: "40126b833a4642c7b8567ae1c9c717ea"
format_stamp: "Formatted at 2025-10-23 08:00:52 on dist-test-slave-wkn7"
I20251023 08:00:52.713954 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1761206396810293-18906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1761206396810293-18906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1761206396810293-18906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:52.736086 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:52.737040 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:52.741791 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:52.742095 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:52.742406 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:52.742626 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:52.796258 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.129:35497
I20251023 08:00:52.796366 22461 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.129:35497 every 8 connection(s)
I20251023 08:00:52.801474 22462 heartbeater.cc:344] Connected to a master server at 127.18.118.190:46545
I20251023 08:00:52.801849 22462 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:52.802551 22462 heartbeater.cc:507] Master 127.18.118.190:46545 requested a full tablet report, sending...
I20251023 08:00:52.804656 22327 ts_manager.cc:194] Registered new tserver with Master: 40126b833a4642c7b8567ae1c9c717ea (127.18.118.129:35497)
I20251023 08:00:52.805531 18906 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.006023105s
I20251023 08:00:52.806766 22327 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42428
I20251023 08:00:52.837074 22426 tablet_service.cc:1505] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20251023 08:00:52.838248 22426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:52.850365 22472 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea: Bootstrap starting.
I20251023 08:00:52.854631 22472 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:52.858716 22472 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea: No bootstrap required, opened a new log
I20251023 08:00:52.859126 22472 ts_tablet_manager.cc:1403] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea: Time spent bootstrapping tablet: real 0.009s	user 0.007s	sys 0.000s
I20251023 08:00:52.860941 22472 raft_consensus.cc:359] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "40126b833a4642c7b8567ae1c9c717ea" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 35497 } }
I20251023 08:00:52.861379 22472 raft_consensus.cc:385] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:52.861605 22472 raft_consensus.cc:740] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40126b833a4642c7b8567ae1c9c717ea, State: Initialized, Role: FOLLOWER
I20251023 08:00:52.862241 22472 consensus_queue.cc:260] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "40126b833a4642c7b8567ae1c9c717ea" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 35497 } }
I20251023 08:00:52.862687 22472 raft_consensus.cc:399] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 08:00:52.862936 22472 raft_consensus.cc:493] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 08:00:52.863189 22472 raft_consensus.cc:3060] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:52.867916 22472 raft_consensus.cc:515] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "40126b833a4642c7b8567ae1c9c717ea" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 35497 } }
I20251023 08:00:52.868445 22472 leader_election.cc:304] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40126b833a4642c7b8567ae1c9c717ea; no voters: 
I20251023 08:00:52.869585 22472 leader_election.cc:290] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea [CANDIDATE]: Term 1 election: Requested vote from peers 
I20251023 08:00:52.869903 22474 raft_consensus.cc:2804] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:52.871229 22474 raft_consensus.cc:697] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea [term 1 LEADER]: Becoming Leader. State: Replica: 40126b833a4642c7b8567ae1c9c717ea, State: Running, Role: LEADER
I20251023 08:00:52.871385 22462 heartbeater.cc:499] Master 127.18.118.190:46545 was elected leader, sending a full tablet report...
I20251023 08:00:52.871938 22474 consensus_queue.cc:237] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "40126b833a4642c7b8567ae1c9c717ea" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 35497 } }
I20251023 08:00:52.872519 22472 ts_tablet_manager.cc:1434] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.005s
I20251023 08:00:52.876818 22476 tablet_replica.cc:442] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea: TxnStatusTablet state changed. Reason: New leader 40126b833a4642c7b8567ae1c9c717ea. Latest consensus state: current_term: 1 leader_uuid: "40126b833a4642c7b8567ae1c9c717ea" committed_config { opid_index: -1 peers { permanent_uuid: "40126b833a4642c7b8567ae1c9c717ea" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 35497 } } }
I20251023 08:00:52.876660 22475 tablet_replica.cc:442] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40126b833a4642c7b8567ae1c9c717ea" committed_config { opid_index: -1 peers { permanent_uuid: "40126b833a4642c7b8567ae1c9c717ea" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 35497 } } }
I20251023 08:00:52.877532 22476 tablet_replica.cc:445] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea: This TxnStatusTablet replica's current role is: LEADER
I20251023 08:00:52.877609 22475 tablet_replica.cc:445] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea: This TxnStatusTablet replica's current role is: LEADER
W20251023 08:00:52.879426 22327 catalog_manager.cc:5482] Ignoring report from unknown tablet 11111111111111111111111111111111
I20251023 08:00:52.881631 22479 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20251023 08:00:52.882015 22479 txn_status_manager.cc:930] Loading transaction status metadata into memory...
W20251023 08:00:52.884002 22327 catalog_manager.cc:5482] Ignoring report from unknown tablet 11111111111111111111111111111111
I20251023 08:00:52.907234 22426 tablet_service.cc:1505] Processing CreateTablet for tablet 22222222222222222222222222222222 (table=22222222222222222222222222222222_table_name [id=22222222222222222222222222222222_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20251023 08:00:52.908280 22426 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22222222222222222222222222222222. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:52.918257 22472 tablet_bootstrap.cc:492] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea: Bootstrap starting.
I20251023 08:00:52.922263 22472 tablet_bootstrap.cc:654] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:52.926038 22472 tablet_bootstrap.cc:492] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea: No bootstrap required, opened a new log
I20251023 08:00:52.926416 22472 ts_tablet_manager.cc:1403] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea: Time spent bootstrapping tablet: real 0.008s	user 0.001s	sys 0.006s
I20251023 08:00:52.928218 22472 raft_consensus.cc:359] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "40126b833a4642c7b8567ae1c9c717ea" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 35497 } }
I20251023 08:00:52.928624 22472 raft_consensus.cc:385] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:52.928853 22472 raft_consensus.cc:740] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40126b833a4642c7b8567ae1c9c717ea, State: Initialized, Role: FOLLOWER
I20251023 08:00:52.929421 22472 consensus_queue.cc:260] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "40126b833a4642c7b8567ae1c9c717ea" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 35497 } }
I20251023 08:00:52.929884 22472 raft_consensus.cc:399] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 08:00:52.930105 22472 raft_consensus.cc:493] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 08:00:52.930358 22472 raft_consensus.cc:3060] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:52.934770 22472 raft_consensus.cc:515] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "40126b833a4642c7b8567ae1c9c717ea" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 35497 } }
I20251023 08:00:52.935294 22472 leader_election.cc:304] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40126b833a4642c7b8567ae1c9c717ea; no voters: 
I20251023 08:00:52.935750 22472 leader_election.cc:290] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea [CANDIDATE]: Term 1 election: Requested vote from peers 
I20251023 08:00:52.935952 22476 raft_consensus.cc:2804] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:52.936424 22476 raft_consensus.cc:697] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea [term 1 LEADER]: Becoming Leader. State: Replica: 40126b833a4642c7b8567ae1c9c717ea, State: Running, Role: LEADER
I20251023 08:00:52.937289 22472 ts_tablet_manager.cc:1434] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea: Time spent starting tablet: real 0.011s	user 0.001s	sys 0.011s
I20251023 08:00:52.937116 22476 consensus_queue.cc:237] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "40126b833a4642c7b8567ae1c9c717ea" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 35497 } }
W20251023 08:00:52.942023 22327 catalog_manager.cc:5482] Ignoring report from unknown tablet 22222222222222222222222222222222
W20251023 08:00:52.945784 22327 catalog_manager.cc:5482] Ignoring report from unknown tablet 22222222222222222222222222222222
I20251023 08:00:52.969851 18906 tablet_server.cc:178] TabletServer@127.18.118.129:0 shutting down...
I20251023 08:00:53.000228 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:53.000974 18906 tablet_replica.cc:333] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea: stopping tablet replica
I20251023 08:00:53.001648 18906 raft_consensus.cc:2243] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea [term 1 LEADER]: Raft consensus shutting down.
I20251023 08:00:53.002092 18906 raft_consensus.cc:2272] T 22222222222222222222222222222222 P 40126b833a4642c7b8567ae1c9c717ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:53.005028 18906 tablet_replica.cc:333] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea: stopping tablet replica
I20251023 08:00:53.005550 18906 raft_consensus.cc:2243] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea [term 1 LEADER]: Raft consensus shutting down.
I20251023 08:00:53.005964 18906 raft_consensus.cc:2272] T 11111111111111111111111111111111 P 40126b833a4642c7b8567ae1c9c717ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:53.026453 18906 tablet_server.cc:195] TabletServer@127.18.118.129:0 shutdown complete.
I20251023 08:00:53.038213 18906 master.cc:561] Master@127.18.118.190:46545 shutting down...
I20251023 08:00:53.052351 18906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3 [term 1 LEADER]: Raft consensus shutting down.
I20251023 08:00:53.052951 18906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:53.053376 18906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 15747058c6f749ea8f7761f221bf73c3: stopping tablet replica
I20251023 08:00:53.072274 18906 master.cc:583] Master@127.18.118.190:46545 shutdown complete.
[       OK ] TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors (637 ms)
[----------] 4 tests from TxnStatusTabletManagementTest (3371 ms total)

[----------] 2 tests from FailedTabletsAreReplacedITest
[ RUN      ] FailedTabletsAreReplacedITest.OneReplica/0
I20251023 08:00:53.097389 18906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.18.118.190:45211
I20251023 08:00:53.098428 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:53.103120 22488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:53.104295 22489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:53.105952 22491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:53.106446 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:53.107092 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:53.107281 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:53.107432 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206453107420 us; error 0 us; skew 500 ppm
I20251023 08:00:53.107935 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:53.110239 18906 webserver.cc:492] Webserver started at http://127.18.118.190:41731/ using document root <none> and password file <none>
I20251023 08:00:53.110706 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:53.110873 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:53.111129 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:53.112174 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/master-0-root/instance:
uuid: "a3c2a0fbfc35408a9851a89dc1e95c79"
format_stamp: "Formatted at 2025-10-23 08:00:53 on dist-test-slave-wkn7"
I20251023 08:00:53.116433 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20251023 08:00:53.119748 22496 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:53.120419 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20251023 08:00:53.120679 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/master-0-root
uuid: "a3c2a0fbfc35408a9851a89dc1e95c79"
format_stamp: "Formatted at 2025-10-23 08:00:53 on dist-test-slave-wkn7"
I20251023 08:00:53.120944 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:53.149170 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:53.150141 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:53.186637 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.190:45211
I20251023 08:00:53.186731 22547 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.190:45211 every 8 connection(s)
I20251023 08:00:53.190284 22548 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:53.200361 22548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79: Bootstrap starting.
I20251023 08:00:53.204732 22548 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:53.208369 22548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79: No bootstrap required, opened a new log
I20251023 08:00:53.210227 22548 raft_consensus.cc:359] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c2a0fbfc35408a9851a89dc1e95c79" member_type: VOTER }
I20251023 08:00:53.210582 22548 raft_consensus.cc:385] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:53.210752 22548 raft_consensus.cc:740] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3c2a0fbfc35408a9851a89dc1e95c79, State: Initialized, Role: FOLLOWER
I20251023 08:00:53.211243 22548 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c2a0fbfc35408a9851a89dc1e95c79" member_type: VOTER }
I20251023 08:00:53.211650 22548 raft_consensus.cc:399] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 08:00:53.211833 22548 raft_consensus.cc:493] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 08:00:53.212036 22548 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:53.216135 22548 raft_consensus.cc:515] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c2a0fbfc35408a9851a89dc1e95c79" member_type: VOTER }
I20251023 08:00:53.216583 22548 leader_election.cc:304] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3c2a0fbfc35408a9851a89dc1e95c79; no voters: 
I20251023 08:00:53.217639 22548 leader_election.cc:290] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20251023 08:00:53.217933 22551 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:53.219178 22551 raft_consensus.cc:697] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79 [term 1 LEADER]: Becoming Leader. State: Replica: a3c2a0fbfc35408a9851a89dc1e95c79, State: Running, Role: LEADER
I20251023 08:00:53.219820 22551 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c2a0fbfc35408a9851a89dc1e95c79" member_type: VOTER }
I20251023 08:00:53.220451 22548 sys_catalog.cc:565] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 08:00:53.222779 22553 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3c2a0fbfc35408a9851a89dc1e95c79. Latest consensus state: current_term: 1 leader_uuid: "a3c2a0fbfc35408a9851a89dc1e95c79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c2a0fbfc35408a9851a89dc1e95c79" member_type: VOTER } }
I20251023 08:00:53.222805 22552 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3c2a0fbfc35408a9851a89dc1e95c79" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3c2a0fbfc35408a9851a89dc1e95c79" member_type: VOTER } }
I20251023 08:00:53.223506 22553 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:53.223838 22552 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79 [sys.catalog]: This master's current role is: LEADER
I20251023 08:00:53.231608 22557 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 08:00:53.235931 22557 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 08:00:53.236898 18906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 08:00:53.244364 22557 catalog_manager.cc:1357] Generated new cluster ID: 34556f3b6c634fedb075ec5185fa07e5
I20251023 08:00:53.244650 22557 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 08:00:53.262342 22557 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 08:00:53.263648 22557 catalog_manager.cc:1514] Loading token signing keys...
I20251023 08:00:53.283782 22557 catalog_manager.cc:6022] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79: Generated new TSK 0
I20251023 08:00:53.284384 22557 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 08:00:53.303680 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:53.309152 22569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:53.310233 22570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:53.312054 22572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:53.312227 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:53.313061 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:53.313314 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:53.313473 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206453313457 us; error 0 us; skew 500 ppm
I20251023 08:00:53.313973 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:53.316406 18906 webserver.cc:492] Webserver started at http://127.18.118.129:38305/ using document root <none> and password file <none>
I20251023 08:00:53.316879 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:53.317062 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:53.317395 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:53.318506 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/ts-0-root/instance:
uuid: "332fd16bcd934f8b9b8e56364d144bd6"
format_stamp: "Formatted at 2025-10-23 08:00:53 on dist-test-slave-wkn7"
I20251023 08:00:53.323076 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20251023 08:00:53.326490 22577 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:53.327304 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20251023 08:00:53.327550 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/ts-0-root
uuid: "332fd16bcd934f8b9b8e56364d144bd6"
format_stamp: "Formatted at 2025-10-23 08:00:53 on dist-test-slave-wkn7"
I20251023 08:00:53.327795 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:53.353775 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:53.354691 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:53.356009 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:00:53.358156 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:53.358332 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:53.358558 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:53.358732 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:53.395848 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.129:33457
I20251023 08:00:53.395951 22639 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.129:33457 every 8 connection(s)
I20251023 08:00:53.400269 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:53.407485 22644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:53.408548 22645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:53.411208 22647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:53.412053 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:53.412163 22640 heartbeater.cc:344] Connected to a master server at 127.18.118.190:45211
I20251023 08:00:53.412617 22640 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:53.413015 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:53.413288 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:53.413323 22640 heartbeater.cc:507] Master 127.18.118.190:45211 requested a full tablet report, sending...
I20251023 08:00:53.413502 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206453413485 us; error 0 us; skew 500 ppm
I20251023 08:00:53.414121 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:53.415810 22513 ts_manager.cc:194] Registered new tserver with Master: 332fd16bcd934f8b9b8e56364d144bd6 (127.18.118.129:33457)
I20251023 08:00:53.417261 18906 webserver.cc:492] Webserver started at http://127.18.118.130:35113/ using document root <none> and password file <none>
I20251023 08:00:53.417696 22513 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36514
I20251023 08:00:53.417775 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:53.418129 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:53.418395 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:53.419497 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/ts-1-root/instance:
uuid: "e2b5b8655ea34e38984eddbf12f77258"
format_stamp: "Formatted at 2025-10-23 08:00:53 on dist-test-slave-wkn7"
I20251023 08:00:53.424006 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20251023 08:00:53.427238 22652 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:53.427995 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20251023 08:00:53.428277 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/ts-1-root
uuid: "e2b5b8655ea34e38984eddbf12f77258"
format_stamp: "Formatted at 2025-10-23 08:00:53 on dist-test-slave-wkn7"
I20251023 08:00:53.428539 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:53.443645 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:53.444559 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:53.445920 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:00:53.448031 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:53.448226 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:53.448458 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:53.448608 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:53.487133 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.130:46389
I20251023 08:00:53.487227 22714 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.130:46389 every 8 connection(s)
I20251023 08:00:53.491770 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:00:53.497923 22718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:53.498693 22719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:00:53.502269 22721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:00:53.502853 18906 server_base.cc:1047] running on GCE node
I20251023 08:00:53.503407 22715 heartbeater.cc:344] Connected to a master server at 127.18.118.190:45211
I20251023 08:00:53.503743 22715 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:53.503822 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:00:53.504086 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:00:53.504253 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206453504234 us; error 0 us; skew 500 ppm
I20251023 08:00:53.504453 22715 heartbeater.cc:507] Master 127.18.118.190:45211 requested a full tablet report, sending...
I20251023 08:00:53.504810 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:00:53.506500 22513 ts_manager.cc:194] Registered new tserver with Master: e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130:46389)
I20251023 08:00:53.507555 18906 webserver.cc:492] Webserver started at http://127.18.118.131:40283/ using document root <none> and password file <none>
I20251023 08:00:53.508193 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:00:53.508433 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:00:53.508371 22513 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36526
I20251023 08:00:53.508915 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:00:53.510116 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/ts-2-root/instance:
uuid: "7c6da85b15a34caf91dc580c9ce415a7"
format_stamp: "Formatted at 2025-10-23 08:00:53 on dist-test-slave-wkn7"
I20251023 08:00:53.514292 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20251023 08:00:53.517835 22726 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:53.518764 18906 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.002s
I20251023 08:00:53.519071 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/ts-2-root
uuid: "7c6da85b15a34caf91dc580c9ce415a7"
format_stamp: "Formatted at 2025-10-23 08:00:53 on dist-test-slave-wkn7"
I20251023 08:00:53.519356 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:00:53.531219 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:00:53.532248 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:00:53.533666 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:00:53.535748 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:00:53.535944 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:53.536175 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:00:53.536329 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:00:53.573429 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.131:34855
I20251023 08:00:53.573534 22788 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.131:34855 every 8 connection(s)
I20251023 08:00:53.586156 22789 heartbeater.cc:344] Connected to a master server at 127.18.118.190:45211
I20251023 08:00:53.586540 22789 heartbeater.cc:461] Registering TS with master...
I20251023 08:00:53.587191 22789 heartbeater.cc:507] Master 127.18.118.190:45211 requested a full tablet report, sending...
I20251023 08:00:53.589079 22513 ts_manager.cc:194] Registered new tserver with Master: 7c6da85b15a34caf91dc580c9ce415a7 (127.18.118.131:34855)
I20251023 08:00:53.590107 18906 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013641699s
I20251023 08:00:53.590508 22513 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36538
I20251023 08:00:53.610638 18906 test_util.cc:276] Using random seed: -720689029
I20251023 08:00:53.631150 22513 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36560:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20251023 08:00:53.674218 22754 tablet_service.cc:1505] Processing CreateTablet for tablet 4a15beec19e44e34884d5045b662b12f (DEFAULT_TABLE table=test-workload [id=4519b619212a499baa3c6306aef43f58]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 08:00:53.675820 22754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a15beec19e44e34884d5045b662b12f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:53.677526 22680 tablet_service.cc:1505] Processing CreateTablet for tablet 4a15beec19e44e34884d5045b662b12f (DEFAULT_TABLE table=test-workload [id=4519b619212a499baa3c6306aef43f58]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 08:00:53.678844 22680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a15beec19e44e34884d5045b662b12f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:53.692148 22605 tablet_service.cc:1505] Processing CreateTablet for tablet 4a15beec19e44e34884d5045b662b12f (DEFAULT_TABLE table=test-workload [id=4519b619212a499baa3c6306aef43f58]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 08:00:53.693017 22809 tablet_bootstrap.cc:492] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7: Bootstrap starting.
I20251023 08:00:53.693668 22605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a15beec19e44e34884d5045b662b12f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:53.698684 22809 tablet_bootstrap.cc:654] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:53.700589 22810 tablet_bootstrap.cc:492] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: Bootstrap starting.
I20251023 08:00:53.703248 22809 tablet_bootstrap.cc:492] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7: No bootstrap required, opened a new log
I20251023 08:00:53.703649 22809 ts_tablet_manager.cc:1403] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7: Time spent bootstrapping tablet: real 0.011s	user 0.009s	sys 0.000s
I20251023 08:00:53.706955 22809 raft_consensus.cc:359] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } } peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } }
I20251023 08:00:53.708036 22810 tablet_bootstrap.cc:654] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:53.707964 22809 raft_consensus.cc:385] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:53.708663 22809 raft_consensus.cc:740] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c6da85b15a34caf91dc580c9ce415a7, State: Initialized, Role: FOLLOWER
I20251023 08:00:53.709406 22809 consensus_queue.cc:260] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } } peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } }
I20251023 08:00:53.711376 22789 heartbeater.cc:499] Master 127.18.118.190:45211 was elected leader, sending a full tablet report...
I20251023 08:00:53.719137 22814 tablet_bootstrap.cc:492] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6: Bootstrap starting.
I20251023 08:00:53.718114 22809 ts_tablet_manager.cc:1434] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20251023 08:00:53.720536 22810 tablet_bootstrap.cc:492] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: No bootstrap required, opened a new log
I20251023 08:00:53.721225 22810 ts_tablet_manager.cc:1403] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: Time spent bootstrapping tablet: real 0.021s	user 0.005s	sys 0.012s
I20251023 08:00:53.723697 22810 raft_consensus.cc:359] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } } peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } }
I20251023 08:00:53.724231 22810 raft_consensus.cc:385] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:53.724462 22810 raft_consensus.cc:740] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b5b8655ea34e38984eddbf12f77258, State: Initialized, Role: FOLLOWER
I20251023 08:00:53.725044 22810 consensus_queue.cc:260] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } } peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } }
I20251023 08:00:53.726998 22715 heartbeater.cc:499] Master 127.18.118.190:45211 was elected leader, sending a full tablet report...
I20251023 08:00:53.727653 22810 ts_tablet_manager.cc:1434] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20251023 08:00:53.727736 22814 tablet_bootstrap.cc:654] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6: Neither blocks nor log segments found. Creating new log.
I20251023 08:00:53.732807 22814 tablet_bootstrap.cc:492] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6: No bootstrap required, opened a new log
I20251023 08:00:53.733316 22814 ts_tablet_manager.cc:1403] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20251023 08:00:53.735266 22814 raft_consensus.cc:359] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } } peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } }
I20251023 08:00:53.735766 22814 raft_consensus.cc:385] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:00:53.735992 22814 raft_consensus.cc:740] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 332fd16bcd934f8b9b8e56364d144bd6, State: Initialized, Role: FOLLOWER
I20251023 08:00:53.736522 22814 consensus_queue.cc:260] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } } peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } }
I20251023 08:00:53.738260 22640 heartbeater.cc:499] Master 127.18.118.190:45211 was elected leader, sending a full tablet report...
I20251023 08:00:53.738958 22814 ts_tablet_manager.cc:1434] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20251023 08:00:53.834982 22815 raft_consensus.cc:493] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 08:00:53.835650 22815 raft_consensus.cc:515] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } } peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } }
I20251023 08:00:53.837850 22815 leader_election.cc:290] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c6da85b15a34caf91dc580c9ce415a7 (127.18.118.131:34855), 332fd16bcd934f8b9b8e56364d144bd6 (127.18.118.129:33457)
I20251023 08:00:53.847368 22764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4a15beec19e44e34884d5045b662b12f" candidate_uuid: "e2b5b8655ea34e38984eddbf12f77258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c6da85b15a34caf91dc580c9ce415a7" is_pre_election: true
I20251023 08:00:53.848140 22764 raft_consensus.cc:2468] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b5b8655ea34e38984eddbf12f77258 in term 0.
I20251023 08:00:53.848857 22615 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4a15beec19e44e34884d5045b662b12f" candidate_uuid: "e2b5b8655ea34e38984eddbf12f77258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "332fd16bcd934f8b9b8e56364d144bd6" is_pre_election: true
I20251023 08:00:53.849182 22656 leader_election.cc:304] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c6da85b15a34caf91dc580c9ce415a7, e2b5b8655ea34e38984eddbf12f77258; no voters: 
I20251023 08:00:53.849511 22615 raft_consensus.cc:2468] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e2b5b8655ea34e38984eddbf12f77258 in term 0.
I20251023 08:00:53.849956 22815 raft_consensus.cc:2804] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 08:00:53.850258 22815 raft_consensus.cc:493] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 08:00:53.850521 22815 raft_consensus.cc:3060] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:53.855326 22815 raft_consensus.cc:515] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } } peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } }
I20251023 08:00:53.856931 22815 leader_election.cc:290] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [CANDIDATE]: Term 1 election: Requested vote from peers 7c6da85b15a34caf91dc580c9ce415a7 (127.18.118.131:34855), 332fd16bcd934f8b9b8e56364d144bd6 (127.18.118.129:33457)
I20251023 08:00:53.857748 22764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4a15beec19e44e34884d5045b662b12f" candidate_uuid: "e2b5b8655ea34e38984eddbf12f77258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c6da85b15a34caf91dc580c9ce415a7"
I20251023 08:00:53.858068 22615 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4a15beec19e44e34884d5045b662b12f" candidate_uuid: "e2b5b8655ea34e38984eddbf12f77258" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "332fd16bcd934f8b9b8e56364d144bd6"
I20251023 08:00:53.858214 22764 raft_consensus.cc:3060] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:53.858543 22615 raft_consensus.cc:3060] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:00:53.863062 22615 raft_consensus.cc:2468] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b5b8655ea34e38984eddbf12f77258 in term 1.
I20251023 08:00:53.863459 22764 raft_consensus.cc:2468] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e2b5b8655ea34e38984eddbf12f77258 in term 1.
I20251023 08:00:53.864017 22653 leader_election.cc:304] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 332fd16bcd934f8b9b8e56364d144bd6, e2b5b8655ea34e38984eddbf12f77258; no voters: 
I20251023 08:00:53.864785 22815 raft_consensus.cc:2804] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:00:53.865795 22815 raft_consensus.cc:697] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [term 1 LEADER]: Becoming Leader. State: Replica: e2b5b8655ea34e38984eddbf12f77258, State: Running, Role: LEADER
I20251023 08:00:53.866971 22815 consensus_queue.cc:237] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } } peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } }
I20251023 08:00:53.873622 22513 catalog_manager.cc:5649] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 reported cstate change: term changed from 0 to 1, leader changed from <none> to e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130). New cstate: current_term: 1 leader_uuid: "e2b5b8655ea34e38984eddbf12f77258" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } } peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } } }
I20251023 08:00:54.145852 22754 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:00:54.145902 22605 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:00:54.148021 22680 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251023 08:00:54.361408 22815 consensus_queue.cc:1048] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Master Summary
               UUID               |       Address        | Status
----------------------------------+----------------------+---------
 a3c2a0fbfc35408a9851a89dc1e95c79 | 127.18.118.190:45211 | HEALTHY

Unusual flags for Master:
                   Flag                   | Value |         Tags          |         Master
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |       Address        | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
 332fd16bcd934f8b9b8e56364d144bd6 | 127.18.118.129:33457 | HEALTHY | <none>   |       0        |       0
 7c6da85b15a34caf91dc580c9ce415a7 | 127.18.118.131:34855 | HEALTHY | <none>   |       0        |       0
 e2b5b8655ea34e38984eddbf12f77258 | 127.18.118.130:46389 | HEALTHY | <none>   |       1        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   | Value |         Tags          |      Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 3 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
 time_source         | system_unsync                                               | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
Tablet 4a15beec19e44e34884d5045b662b12f of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
  e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130:46389): RUNNING [LEADER]
  7c6da85b15a34caf91dc580c9ce415a7 (127.18.118.131:34855): RUNNING
  332fd16bcd934f8b9b8e56364d144bd6 (127.18.118.129:33457): RUNNING
All reported replicas are:
  A = e2b5b8655ea34e38984eddbf12f77258
  B = 7c6da85b15a34caf91dc580c9ce415a7
  C = 332fd16bcd934f8b9b8e56364d144bd6
The consensus matrix is:
 Config source |   Replicas   | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
 master        | A*  B   C    |              |              | Yes
 A             | A*  B   C    | 1            | -1           | Yes
 B             | A   B   C    | 1            | -1           | Yes
 C             | A   B   C    | 1            | -1           | Yes

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |       Status       | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | CONSENSUS_MISMATCH | 1             | 0       | 0          | 0                | 1

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20251023 08:00:54.379984 22815 consensus_queue.cc:1048] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [LEADER]: Connected to new peer: Peer: permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 08:00:54.380839 18906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20251023 08:00:54.741470 22605 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:00:54.749907 22754 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:00:54.750131 22680 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
Master Summary
               UUID               |       Address        | Status
----------------------------------+----------------------+---------
 a3c2a0fbfc35408a9851a89dc1e95c79 | 127.18.118.190:45211 | HEALTHY

Unusual flags for Master:
                   Flag                   | Value |         Tags          |         Master
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |       Address        | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
 332fd16bcd934f8b9b8e56364d144bd6 | 127.18.118.129:33457 | HEALTHY | <none>   |       0        |       0
 7c6da85b15a34caf91dc580c9ce415a7 | 127.18.118.131:34855 | HEALTHY | <none>   |       0        |       0
 e2b5b8655ea34e38984eddbf12f77258 | 127.18.118.130:46389 | HEALTHY | <none>   |       1        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   | Value |         Tags          |      Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 3 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
 time_source         | system_unsync                                               | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
     Name      | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20251023 08:00:54.949585 18906 log_verifier.cc:126] Checking tablet 4a15beec19e44e34884d5045b662b12f
I20251023 08:00:55.009850 18906 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 4a15beec19e44e34884d5045b662b12f
I20251023 08:00:55.010406 18906 tablet_replica.cc:333] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: stopping tablet replica
I20251023 08:00:55.010938 18906 raft_consensus.cc:2243] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [term 1 LEADER]: Raft consensus shutting down.
I20251023 08:00:55.011719 18906 raft_consensus.cc:2272] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:55.219398 22605 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:00:55.221904 22754 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:00:55.222292 22680 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |       Address        | Status
----------------------------------+----------------------+---------
 a3c2a0fbfc35408a9851a89dc1e95c79 | 127.18.118.190:45211 | HEALTHY

Unusual flags for Master:
                   Flag                   | Value |         Tags          |         Master
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |       Address        | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
 332fd16bcd934f8b9b8e56364d144bd6 | 127.18.118.129:33457 | HEALTHY | <none>   |       0        |       0
 7c6da85b15a34caf91dc580c9ce415a7 | 127.18.118.131:34855 | HEALTHY | <none>   |       0        |       0
 e2b5b8655ea34e38984eddbf12f77258 | 127.18.118.130:46389 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   | Value |         Tags          |      Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 3 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
 time_source         | system_unsync                                               | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
Tablet 4a15beec19e44e34884d5045b662b12f of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
  e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130:46389): not running [LEADER]
    State:       FAILED
    Data state:  TABLET_DATA_READY
    Last status: IO error: INJECTED ERROR: tablet failed
  7c6da85b15a34caf91dc580c9ce415a7 (127.18.118.131:34855): RUNNING
  332fd16bcd934f8b9b8e56364d144bd6 (127.18.118.129:33457): RUNNING
All reported replicas are:
  A = e2b5b8655ea34e38984eddbf12f77258
  B = 7c6da85b15a34caf91dc580c9ce415a7
  C = 332fd16bcd934f8b9b8e56364d144bd6
The consensus matrix is:
 Config source |        Replicas        | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
 master        | A*  B   C              |              |              | Yes
 A             | [config not available] |              |              | 
 B             | A*  B   C              | 1            | -1           | Yes
 C             | A*  B   C              | 1            | -1           | Yes

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |      Status      | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | UNDER_REPLICATED | 1             | 0       | 0          | 1                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20251023 08:00:55.435698 18906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20251023 08:00:55.794670 22605 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:00:55.797091 22680 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:00:55.798965 22754 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |       Address        | Status
----------------------------------+----------------------+---------
 a3c2a0fbfc35408a9851a89dc1e95c79 | 127.18.118.190:45211 | HEALTHY

Unusual flags for Master:
                   Flag                   | Value |         Tags          |         Master
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |       Address        | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
 332fd16bcd934f8b9b8e56364d144bd6 | 127.18.118.129:33457 | HEALTHY | <none>   |       0        |       0
 7c6da85b15a34caf91dc580c9ce415a7 | 127.18.118.131:34855 | HEALTHY | <none>   |       0        |       0
 e2b5b8655ea34e38984eddbf12f77258 | 127.18.118.130:46389 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   | Value |         Tags          |      Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 3 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
 time_source         | system_unsync                                               | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
Tablet 4a15beec19e44e34884d5045b662b12f of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
  e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130:46389): not running [LEADER]
    State:       FAILED
    Data state:  TABLET_DATA_READY
    Last status: IO error: INJECTED ERROR: tablet failed
  7c6da85b15a34caf91dc580c9ce415a7 (127.18.118.131:34855): RUNNING
  332fd16bcd934f8b9b8e56364d144bd6 (127.18.118.129:33457): RUNNING
All reported replicas are:
  A = e2b5b8655ea34e38984eddbf12f77258
  B = 7c6da85b15a34caf91dc580c9ce415a7
  C = 332fd16bcd934f8b9b8e56364d144bd6
The consensus matrix is:
 Config source |        Replicas        | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
 master        | A*  B   C              |              |              | Yes
 A             | [config not available] |              |              | 
 B             | A*  B   C              | 1            | -1           | Yes
 C             | A*  B   C              | 1            | -1           | Yes

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |      Status      | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | UNDER_REPLICATED | 1             | 0       | 0          | 1                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20251023 08:00:56.007462 18906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20251023 08:00:56.423777 22957 raft_consensus.cc:493] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b5b8655ea34e38984eddbf12f77258)
I20251023 08:00:56.424554 22957 raft_consensus.cc:515] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } } peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } }
I20251023 08:00:56.435784 22957 leader_election.cc:290] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130:46389), 332fd16bcd934f8b9b8e56364d144bd6 (127.18.118.129:33457)
I20251023 08:00:56.448361 22754 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:00:56.449126 22963 raft_consensus.cc:493] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader e2b5b8655ea34e38984eddbf12f77258)
I20251023 08:00:56.449690 22963 raft_consensus.cc:515] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } } peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } }
I20251023 08:00:56.459749 22963 leader_election.cc:290] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130:46389), 7c6da85b15a34caf91dc580c9ce415a7 (127.18.118.131:34855)
I20251023 08:00:56.463132 22690 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4a15beec19e44e34884d5045b662b12f" candidate_uuid: "7c6da85b15a34caf91dc580c9ce415a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b5b8655ea34e38984eddbf12f77258" is_pre_election: true
W20251023 08:00:56.464924 22729 leader_election.cc:343] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130:46389): Illegal state: cannot vote while shut down
I20251023 08:00:56.476052 22680 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:00:56.483196 22615 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4a15beec19e44e34884d5045b662b12f" candidate_uuid: "7c6da85b15a34caf91dc580c9ce415a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "332fd16bcd934f8b9b8e56364d144bd6" is_pre_election: true
I20251023 08:00:56.483636 22605 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:00:56.484145 22615 raft_consensus.cc:2468] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c6da85b15a34caf91dc580c9ce415a7 in term 1.
I20251023 08:00:56.486353 22727 leader_election.cc:304] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 332fd16bcd934f8b9b8e56364d144bd6, 7c6da85b15a34caf91dc580c9ce415a7; no voters: e2b5b8655ea34e38984eddbf12f77258
I20251023 08:00:56.487867 22957 raft_consensus.cc:2804] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251023 08:00:56.488229 22957 raft_consensus.cc:493] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader e2b5b8655ea34e38984eddbf12f77258)
I20251023 08:00:56.488581 22957 raft_consensus.cc:3060] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:56.494776 22764 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4a15beec19e44e34884d5045b662b12f" candidate_uuid: "332fd16bcd934f8b9b8e56364d144bd6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "7c6da85b15a34caf91dc580c9ce415a7" is_pre_election: true
I20251023 08:00:56.496421 22690 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4a15beec19e44e34884d5045b662b12f" candidate_uuid: "332fd16bcd934f8b9b8e56364d144bd6" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b5b8655ea34e38984eddbf12f77258" is_pre_election: true
W20251023 08:00:56.497704 22580 leader_election.cc:343] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130:46389): Illegal state: cannot vote while shut down
I20251023 08:00:56.498155 22957 raft_consensus.cc:515] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } } peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } }
I20251023 08:00:56.499279 22764 raft_consensus.cc:2393] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 332fd16bcd934f8b9b8e56364d144bd6 in current term 2: Already voted for candidate 7c6da85b15a34caf91dc580c9ce415a7 in this term.
I20251023 08:00:56.500597 22957 leader_election.cc:290] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [CANDIDATE]: Term 2 election: Requested vote from peers e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130:46389), 332fd16bcd934f8b9b8e56364d144bd6 (127.18.118.129:33457)
I20251023 08:00:56.500969 22690 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4a15beec19e44e34884d5045b662b12f" candidate_uuid: "7c6da85b15a34caf91dc580c9ce415a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "e2b5b8655ea34e38984eddbf12f77258"
I20251023 08:00:56.501960 22615 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4a15beec19e44e34884d5045b662b12f" candidate_uuid: "7c6da85b15a34caf91dc580c9ce415a7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "332fd16bcd934f8b9b8e56364d144bd6"
W20251023 08:00:56.502177 22729 leader_election.cc:343] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130:46389): Illegal state: cannot vote while shut down
I20251023 08:00:56.502521 22615 raft_consensus.cc:3060] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [term 1 FOLLOWER]: Advancing to term 2
I20251023 08:00:56.502363 22581 leader_election.cc:304] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 332fd16bcd934f8b9b8e56364d144bd6; no voters: 7c6da85b15a34caf91dc580c9ce415a7, e2b5b8655ea34e38984eddbf12f77258
I20251023 08:00:56.509655 22615 raft_consensus.cc:2468] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c6da85b15a34caf91dc580c9ce415a7 in term 2.
I20251023 08:00:56.510206 22963 raft_consensus.cc:2749] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251023 08:00:56.511520 22727 leader_election.cc:304] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 332fd16bcd934f8b9b8e56364d144bd6, 7c6da85b15a34caf91dc580c9ce415a7; no voters: e2b5b8655ea34e38984eddbf12f77258
I20251023 08:00:56.512466 22957 raft_consensus.cc:2804] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [term 2 FOLLOWER]: Leader election won for term 2
I20251023 08:00:56.513885 22957 raft_consensus.cc:697] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [term 2 LEADER]: Becoming Leader. State: Replica: 7c6da85b15a34caf91dc580c9ce415a7, State: Running, Role: LEADER
I20251023 08:00:56.514806 22957 consensus_queue.cc:237] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } } peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } }
I20251023 08:00:56.522387 22511 catalog_manager.cc:5649] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 reported cstate change: term changed from 1 to 2, leader changed from e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130) to 7c6da85b15a34caf91dc580c9ce415a7 (127.18.118.131). New cstate: current_term: 2 leader_uuid: "7c6da85b15a34caf91dc580c9ce415a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } } peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } } }
Master Summary
               UUID               |       Address        | Status
----------------------------------+----------------------+---------
 a3c2a0fbfc35408a9851a89dc1e95c79 | 127.18.118.190:45211 | HEALTHY

Unusual flags for Master:
                   Flag                   | Value |         Tags          |         Master
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |       Address        | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
 332fd16bcd934f8b9b8e56364d144bd6 | 127.18.118.129:33457 | HEALTHY | <none>   |       0        |       0
 7c6da85b15a34caf91dc580c9ce415a7 | 127.18.118.131:34855 | HEALTHY | <none>   |       0        |       0
 e2b5b8655ea34e38984eddbf12f77258 | 127.18.118.130:46389 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   | Value |         Tags          |      Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 3 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
 time_source         | system_unsync                                               | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
Tablet 4a15beec19e44e34884d5045b662b12f of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
  e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130:46389): not running [LEADER]
    State:       FAILED
    Data state:  TABLET_DATA_READY
    Last status: IO error: INJECTED ERROR: tablet failed
  7c6da85b15a34caf91dc580c9ce415a7 (127.18.118.131:34855): RUNNING
  332fd16bcd934f8b9b8e56364d144bd6 (127.18.118.129:33457): RUNNING
All reported replicas are:
  A = e2b5b8655ea34e38984eddbf12f77258
  B = 7c6da85b15a34caf91dc580c9ce415a7
  C = 332fd16bcd934f8b9b8e56364d144bd6
The consensus matrix is:
 Config source |        Replicas        | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
 master        | A*  B   C              |              |              | Yes
 A             | [config not available] |              |              | 
 B             | A*  B   C              | 1            | -1           | Yes
 C             | A*  B   C              | 1            | -1           | Yes

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |      Status      | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | UNDER_REPLICATED | 1             | 0       | 0          | 1                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20251023 08:00:56.703807 18906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
W20251023 08:00:57.034749 22729 consensus_peers.cc:597] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 -> Peer e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130:46389): Couldn't send request to peer e2b5b8655ea34e38984eddbf12f77258. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20251023 08:00:57.126065 22615 raft_consensus.cc:1275] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [term 2 FOLLOWER]: Refusing update from remote peer 7c6da85b15a34caf91dc580c9ce415a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20251023 08:00:57.127461 22973 consensus_queue.cc:1048] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 08:00:57.272720 22605 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:00:57.327318 22754 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251023 08:00:57.332180 22680 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:00:57.538628 22973 raft_consensus.cc:1012] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7: Attempting to remove follower e2b5b8655ea34e38984eddbf12f77258 from the Raft config. Reason: The tablet replica hosted on peer e2b5b8655ea34e38984eddbf12f77258 has failed
Master Summary
               UUID               |       Address        | Status
----------------------------------+----------------------+---------
 a3c2a0fbfc35408a9851a89dc1e95c79 | 127.18.118.190:45211 | HEALTHY

Unusual flags for Master:
I20251023 08:00:57.541713 22973 consensus_queue.cc:237] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } }
                   Flag                   | Value |         Tags          |         Master
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |       Address        | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
 332fd16bcd934f8b9b8e56364d144bd6 | 127.18.118.129:33457 | HEALTHY | <none>   |       0        |       0
 7c6da85b15a34caf91dc580c9ce415a7 | 127.18.118.131:34855 | HEALTHY | <none>   |       1        |       0
 e2b5b8655ea34e38984eddbf12f77258 | 127.18.118.130:46389 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   | Value |         Tags          |      Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 3 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
 time_source         | system_unsync                                               | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
Tablet 4a15beec19e44e34884d5045b662b12f of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
  e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130:46389): not running
    State:       FAILED
    Data state:  TABLET_DATA_READY
    Last status: IO error: INJECTED ERROR: tablet failed
  7c6da85b15a34caf91dc580c9ce415a7 (127.18.118.131:34855): RUNNING [LEADER]
  332fd16bcd934f8b9b8e56364d144bd6 (127.18.118.129:33457): RUNNING
All reported replicas are:
  A = e2b5b8655ea34e38984eddbf12f77258
  B = 7c6da85b15a34caf91dc580c9ce415a7
  C = 332fd16bcd934f8b9b8e56364d144bd6
The consensus matrix is:
 Config source |        Replicas        | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
 master        | A   B*  C              |              |              | Yes
 A             | [config not available] |              |              | 
 B             | A   B*  C              | 2            | -1           | Yes
 C             | A   B*  C              | 2            | -1           | Yes
I20251023 08:00:57.547837 22615 raft_consensus.cc:1275] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [term 2 FOLLOWER]: Refusing update from remote peer 7c6da85b15a34caf91dc580c9ce415a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |      Status      | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | UNDER_REPLICATED | 1             | 0       | 0          | 1                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
I20251023 08:00:57.549499 22973 consensus_queue.cc:1048] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20251023 08:00:57.556644 18906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20251023 08:00:57.560025 23007 raft_consensus.cc:2955] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, VOTER e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } } }
I20251023 08:00:57.561805 22615 raft_consensus.cc:2955] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, VOTER e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130) evicted. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } } }
I20251023 08:00:57.574980 22513 catalog_manager.cc:5649] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 reported cstate change: config changed from index -1 to 3, VOTER e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130) evicted. New cstate: current_term: 2 leader_uuid: "7c6da85b15a34caf91dc580c9ce415a7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } } }
I20251023 08:00:57.587894 22680 tablet_service.cc:1552] Processing DeleteTablet for tablet 4a15beec19e44e34884d5045b662b12f with delete_type TABLET_DATA_TOMBSTONED (TS e2b5b8655ea34e38984eddbf12f77258 not found in new config with opid_index 3) from {username='slave'} at 127.0.0.1:45800
I20251023 08:00:57.590368 23012 ts_tablet_manager.cc:1916] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 08:00:57.591414 22764 consensus_queue.cc:237] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } } peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } }
I20251023 08:00:57.596323 22615 raft_consensus.cc:1275] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [term 2 FOLLOWER]: Refusing update from remote peer 7c6da85b15a34caf91dc580c9ce415a7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
W20251023 08:00:57.597069 22729 consensus_peers.cc:597] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 -> Peer e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130:46389): Couldn't send request to peer e2b5b8655ea34e38984eddbf12f77258. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: FAILED. This is attempt 1: this message will repeat every 5th retry.
I20251023 08:00:57.597697 23007 consensus_queue.cc:1048] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20251023 08:00:57.603009 22973 raft_consensus.cc:2955] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } } peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } } }
I20251023 08:00:57.604359 22615 raft_consensus.cc:2955] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } } peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } } }
I20251023 08:00:57.606091 23012 ts_tablet_manager.cc:1929] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20251023 08:00:57.606587 23012 log.cc:1199] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: Deleting WAL directory at /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1761206396810293-18906-0/minicluster-data/ts-1-root/wals/4a15beec19e44e34884d5045b662b12f
I20251023 08:00:57.608417 22499 catalog_manager.cc:4985] TS e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130:46389): tablet 4a15beec19e44e34884d5045b662b12f (table test-workload [id=4519b619212a499baa3c6306aef43f58]) successfully deleted
I20251023 08:00:57.612229 22500 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:VOTER RPC for tablet 4a15beec19e44e34884d5045b662b12f with cas_config_opid_index 3: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20251023 08:00:57.614356 22513 catalog_manager.cc:5649] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 reported cstate change: config changed from index 3 to 4, VOTER e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130) added. New cstate: current_term: 2 leader_uuid: "7c6da85b15a34caf91dc580c9ce415a7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } } peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } } }
I20251023 08:00:58.215404 23026 ts_tablet_manager.cc:933] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: Initiating tablet copy from peer 7c6da85b15a34caf91dc580c9ce415a7 (127.18.118.131:34855)
I20251023 08:00:58.217190 23026 tablet_copy_client.cc:323] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: tablet copy: Beginning tablet copy session from remote peer at address 127.18.118.131:34855
I20251023 08:00:58.218684 22774 tablet_copy_service.cc:140] P 7c6da85b15a34caf91dc580c9ce415a7: Received BeginTabletCopySession request for tablet 4a15beec19e44e34884d5045b662b12f from peer e2b5b8655ea34e38984eddbf12f77258 ({username='slave'} at 127.0.0.1:47262)
I20251023 08:00:58.219214 22774 tablet_copy_service.cc:161] P 7c6da85b15a34caf91dc580c9ce415a7: Beginning new tablet copy session on tablet 4a15beec19e44e34884d5045b662b12f from peer e2b5b8655ea34e38984eddbf12f77258 at {username='slave'} at 127.0.0.1:47262: session id = e2b5b8655ea34e38984eddbf12f77258-4a15beec19e44e34884d5045b662b12f
I20251023 08:00:58.227730 22774 tablet_copy_source_session.cc:215] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 08:00:58.230108 23026 ts_tablet_manager.cc:1916] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: Deleting tablet data with delete state TABLET_DATA_COPYING
I20251023 08:00:58.246618 23026 ts_tablet_manager.cc:1929] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.1
I20251023 08:00:58.247298 23026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a15beec19e44e34884d5045b662b12f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:00:58.253981 23026 tablet_copy_client.cc:806] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: tablet copy: Starting download of 0 data blocks...
I20251023 08:00:58.254498 23026 tablet_copy_client.cc:670] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: tablet copy: Starting download of 1 WAL segments...
I20251023 08:00:58.258059 23026 tablet_copy_client.cc:538] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 08:00:58.265739 23026 tablet_bootstrap.cc:492] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: Bootstrap starting.
I20251023 08:00:58.298101 22605 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:00:58.312603 22754 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251023 08:00:58.314986 22680 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:00:58.317341 23026 tablet_bootstrap.cc:492] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 08:00:58.318311 23026 tablet_bootstrap.cc:492] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: Bootstrap complete.
I20251023 08:00:58.319110 23026 ts_tablet_manager.cc:1403] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: Time spent bootstrapping tablet: real 0.054s	user 0.027s	sys 0.008s
I20251023 08:00:58.321807 23026 raft_consensus.cc:359] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } } peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } }
I20251023 08:00:58.322392 23026 raft_consensus.cc:740] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2b5b8655ea34e38984eddbf12f77258, State: Initialized, Role: FOLLOWER
I20251023 08:00:58.323134 23026 consensus_queue.cc:260] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "7c6da85b15a34caf91dc580c9ce415a7" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 34855 } } peers { permanent_uuid: "332fd16bcd934f8b9b8e56364d144bd6" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 33457 } } peers { permanent_uuid: "e2b5b8655ea34e38984eddbf12f77258" member_type: VOTER last_known_addr { host: "127.18.118.130" port: 46389 } }
I20251023 08:00:58.325853 23026 ts_tablet_manager.cc:1434] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20251023 08:00:58.327543 22774 tablet_copy_service.cc:342] P 7c6da85b15a34caf91dc580c9ce415a7: Request end of tablet copy session e2b5b8655ea34e38984eddbf12f77258-4a15beec19e44e34884d5045b662b12f received from {username='slave'} at 127.0.0.1:47262
I20251023 08:00:58.327976 22774 tablet_copy_service.cc:434] P 7c6da85b15a34caf91dc580c9ce415a7: ending tablet copy session e2b5b8655ea34e38984eddbf12f77258-4a15beec19e44e34884d5045b662b12f on tablet 4a15beec19e44e34884d5045b662b12f with peer e2b5b8655ea34e38984eddbf12f77258
Master Summary
               UUID               |       Address        | Status
----------------------------------+----------------------+---------
 a3c2a0fbfc35408a9851a89dc1e95c79 | 127.18.118.190:45211 | HEALTHY

Unusual flags for Master:
                   Flag                   | Value |         Tags          |         Master
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |       Address        | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
 332fd16bcd934f8b9b8e56364d144bd6 | 127.18.118.129:33457 | HEALTHY | <none>   |       0        |       0
 7c6da85b15a34caf91dc580c9ce415a7 | 127.18.118.131:34855 | HEALTHY | <none>   |       1        |       0
 e2b5b8655ea34e38984eddbf12f77258 | 127.18.118.130:46389 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   | Value |         Tags          |      Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 3 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
 time_source         | system_unsync                                               | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
Tablet 4a15beec19e44e34884d5045b662b12f of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
  7c6da85b15a34caf91dc580c9ce415a7 (127.18.118.131:34855): RUNNING [LEADER]
  332fd16bcd934f8b9b8e56364d144bd6 (127.18.118.129:33457): RUNNING
  e2b5b8655ea34e38984eddbf12f77258 (127.18.118.130:46389): not running
    State:       BOOTSTRAPPING
    Data state:  TABLET_DATA_READY
    Last status: Bootstrap starting.
All reported replicas are:
  A = 7c6da85b15a34caf91dc580c9ce415a7
  B = 332fd16bcd934f8b9b8e56364d144bd6
  C = e2b5b8655ea34e38984eddbf12f77258
The consensus matrix is:
 Config source |   Replicas   | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
 master        | A*  B   C    |              |              | Yes
 A             | A*  B   C    | 2            | 4            | Yes
 B             | A*  B   C    | 2            | 4            | Yes
 C             | A   B   C    | 2            | 4            | Yes

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |      Status      | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | UNDER_REPLICATED | 1             | 0       | 0          | 1                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20251023 08:00:58.534768 18906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20251023 08:00:58.678761 22690 raft_consensus.cc:1217] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 2.3->[2.4-2.4]   Dedup: 2.4->[]
I20251023 08:00:59.522541 22605 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:00:59.523782 22754 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251023 08:00:59.525385 22680 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |       Address        | Status
----------------------------------+----------------------+---------
 a3c2a0fbfc35408a9851a89dc1e95c79 | 127.18.118.190:45211 | HEALTHY

Unusual flags for Master:
                   Flag                   | Value |         Tags          |         Master
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |       Address        | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
 332fd16bcd934f8b9b8e56364d144bd6 | 127.18.118.129:33457 | HEALTHY | <none>   |       0        |       0
 7c6da85b15a34caf91dc580c9ce415a7 | 127.18.118.131:34855 | HEALTHY | <none>   |       1        |       0
 e2b5b8655ea34e38984eddbf12f77258 | 127.18.118.130:46389 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       3

Unusual flags for Tablet Server:
                   Flag                   | Value |         Tags          |      Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
 raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
 rpc_server_allow_ephemeral_ports         | true  | unsafe                | all 3 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
 time_source         | system_unsync                                               | all 3 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 4 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
     Name      | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 1
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 3
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20251023 08:00:59.724699 18906 log_verifier.cc:126] Checking tablet 4a15beec19e44e34884d5045b662b12f
I20251023 08:00:59.784668 18906 log_verifier.cc:177] Verified matching terms for 4 ops in tablet 4a15beec19e44e34884d5045b662b12f
I20251023 08:00:59.787854 18906 tablet_server.cc:178] TabletServer@127.18.118.129:0 shutting down...
I20251023 08:00:59.816838 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:59.817639 18906 tablet_replica.cc:333] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6: stopping tablet replica
I20251023 08:00:59.818200 18906 raft_consensus.cc:2243] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:59.818708 18906 raft_consensus.cc:2272] T 4a15beec19e44e34884d5045b662b12f P 332fd16bcd934f8b9b8e56364d144bd6 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:59.841576 18906 tablet_server.cc:195] TabletServer@127.18.118.129:0 shutdown complete.
I20251023 08:00:59.853289 18906 tablet_server.cc:178] TabletServer@127.18.118.130:0 shutting down...
I20251023 08:00:59.875452 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:59.876174 18906 tablet_replica.cc:333] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258: stopping tablet replica
I20251023 08:00:59.876713 18906 raft_consensus.cc:2243] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251023 08:00:59.877125 18906 raft_consensus.cc:2272] T 4a15beec19e44e34884d5045b662b12f P e2b5b8655ea34e38984eddbf12f77258 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:59.896986 18906 tablet_server.cc:195] TabletServer@127.18.118.130:0 shutdown complete.
I20251023 08:00:59.908041 18906 tablet_server.cc:178] TabletServer@127.18.118.131:0 shutting down...
I20251023 08:00:59.933837 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:00:59.934551 18906 tablet_replica.cc:333] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7: stopping tablet replica
I20251023 08:00:59.935046 18906 raft_consensus.cc:2243] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [term 2 LEADER]: Raft consensus shutting down.
I20251023 08:00:59.935809 18906 raft_consensus.cc:2272] T 4a15beec19e44e34884d5045b662b12f P 7c6da85b15a34caf91dc580c9ce415a7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:59.956434 18906 tablet_server.cc:195] TabletServer@127.18.118.131:0 shutdown complete.
I20251023 08:00:59.967450 18906 master.cc:561] Master@127.18.118.190:45211 shutting down...
I20251023 08:00:59.987123 18906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79 [term 1 LEADER]: Raft consensus shutting down.
I20251023 08:00:59.987659 18906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:00:59.987982 18906 tablet_replica.cc:333] T 00000000000000000000000000000000 P a3c2a0fbfc35408a9851a89dc1e95c79: stopping tablet replica
I20251023 08:01:00.006351 18906 master.cc:583] Master@127.18.118.190:45211 shutdown complete.
[       OK ] FailedTabletsAreReplacedITest.OneReplica/0 (6940 ms)
[ RUN      ] FailedTabletsAreReplacedITest.OneReplica/1
I20251023 08:01:00.037638 18906 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.18.118.190:33813
I20251023 08:01:00.038720 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:01:00.043808 23077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:01:00.044633 23078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:01:00.047031 18906 server_base.cc:1047] running on GCE node
W20251023 08:01:00.047581 23080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:01:00.048353 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:01:00.048538 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:01:00.048689 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206460048672 us; error 0 us; skew 500 ppm
I20251023 08:01:00.049191 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:01:00.051313 18906 webserver.cc:492] Webserver started at http://127.18.118.190:43883/ using document root <none> and password file <none>
I20251023 08:01:00.051759 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:01:00.051929 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:01:00.052171 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:01:00.053236 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/master-0-root/instance:
uuid: "5fe8f347ed5541819166014f37970118"
format_stamp: "Formatted at 2025-10-23 08:01:00 on dist-test-slave-wkn7"
I20251023 08:01:00.057521 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20251023 08:01:00.061131 23085 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:01:00.061995 18906 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20251023 08:01:00.062283 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/master-0-root
uuid: "5fe8f347ed5541819166014f37970118"
format_stamp: "Formatted at 2025-10-23 08:01:00 on dist-test-slave-wkn7"
I20251023 08:01:00.062727 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:01:00.091612 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:01:00.092806 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:01:00.131489 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.190:33813
I20251023 08:01:00.131599 23136 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.190:33813 every 8 connection(s)
I20251023 08:01:00.135190 23137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:01:00.145576 23137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118: Bootstrap starting.
I20251023 08:01:00.150182 23137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118: Neither blocks nor log segments found. Creating new log.
I20251023 08:01:00.154311 23137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118: No bootstrap required, opened a new log
I20251023 08:01:00.156314 23137 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fe8f347ed5541819166014f37970118" member_type: VOTER }
I20251023 08:01:00.156701 23137 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:01:00.156941 23137 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fe8f347ed5541819166014f37970118, State: Initialized, Role: FOLLOWER
I20251023 08:01:00.157548 23137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fe8f347ed5541819166014f37970118" member_type: VOTER }
I20251023 08:01:00.157987 23137 raft_consensus.cc:399] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251023 08:01:00.158191 23137 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251023 08:01:00.158440 23137 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:01:00.163069 23137 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fe8f347ed5541819166014f37970118" member_type: VOTER }
I20251023 08:01:00.163571 23137 leader_election.cc:304] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fe8f347ed5541819166014f37970118; no voters: 
I20251023 08:01:00.164759 23137 leader_election.cc:290] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20251023 08:01:00.165186 23140 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:01:00.166503 23140 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118 [term 1 LEADER]: Becoming Leader. State: Replica: 5fe8f347ed5541819166014f37970118, State: Running, Role: LEADER
I20251023 08:01:00.167063 23140 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fe8f347ed5541819166014f37970118" member_type: VOTER }
I20251023 08:01:00.167706 23137 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118 [sys.catalog]: configured and running, proceeding with master startup.
I20251023 08:01:00.169879 23142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fe8f347ed5541819166014f37970118. Latest consensus state: current_term: 1 leader_uuid: "5fe8f347ed5541819166014f37970118" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fe8f347ed5541819166014f37970118" member_type: VOTER } }
I20251023 08:01:00.169960 23141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5fe8f347ed5541819166014f37970118" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fe8f347ed5541819166014f37970118" member_type: VOTER } }
I20251023 08:01:00.170607 23142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118 [sys.catalog]: This master's current role is: LEADER
I20251023 08:01:00.170725 23141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118 [sys.catalog]: This master's current role is: LEADER
I20251023 08:01:00.173759 23145 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251023 08:01:00.178797 23145 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251023 08:01:00.182181 18906 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251023 08:01:00.186955 23145 catalog_manager.cc:1357] Generated new cluster ID: 0b840e28f3ed42e1812fc0af109fd69d
I20251023 08:01:00.187223 23145 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251023 08:01:00.202576 23145 catalog_manager.cc:1380] Generated new certificate authority record
I20251023 08:01:00.203749 23145 catalog_manager.cc:1514] Loading token signing keys...
I20251023 08:01:00.219048 23145 catalog_manager.cc:6022] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118: Generated new TSK 0
I20251023 08:01:00.219590 23145 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251023 08:01:00.248401 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:01:00.254262 23158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:01:00.255684 23159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:01:00.257040 18906 server_base.cc:1047] running on GCE node
W20251023 08:01:00.256985 23161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:01:00.258232 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:01:00.258410 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:01:00.258538 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206460258528 us; error 0 us; skew 500 ppm
I20251023 08:01:00.258947 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:01:00.261080 18906 webserver.cc:492] Webserver started at http://127.18.118.129:41575/ using document root <none> and password file <none>
I20251023 08:01:00.261534 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:01:00.261683 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:01:00.261889 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:01:00.262851 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-0-root/instance:
uuid: "6c26960f9a354d648d46267ff91e74b2"
format_stamp: "Formatted at 2025-10-23 08:01:00 on dist-test-slave-wkn7"
I20251023 08:01:00.267041 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.002s
I20251023 08:01:00.270138 23166 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:01:00.270866 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20251023 08:01:00.271144 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-0-root
uuid: "6c26960f9a354d648d46267ff91e74b2"
format_stamp: "Formatted at 2025-10-23 08:01:00 on dist-test-slave-wkn7"
I20251023 08:01:00.271390 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:01:00.286988 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:01:00.288138 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:01:00.289757 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:01:00.291932 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:01:00.292129 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:01:00.292375 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:01:00.292531 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:01:00.332237 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.129:41675
I20251023 08:01:00.332345 23228 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.129:41675 every 8 connection(s)
I20251023 08:01:00.337119 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:01:00.344417 23233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:01:00.346113 23234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:01:00.349608 18906 server_base.cc:1047] running on GCE node
W20251023 08:01:00.349848 23236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:01:00.350855 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20251023 08:01:00.350864 23229 heartbeater.cc:344] Connected to a master server at 127.18.118.190:33813
W20251023 08:01:00.351153 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:01:00.351315 23229 heartbeater.cc:461] Registering TS with master...
I20251023 08:01:00.351342 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206460351326 us; error 0 us; skew 500 ppm
I20251023 08:01:00.352075 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:01:00.352119 23229 heartbeater.cc:507] Master 127.18.118.190:33813 requested a full tablet report, sending...
I20251023 08:01:00.354401 23102 ts_manager.cc:194] Registered new tserver with Master: 6c26960f9a354d648d46267ff91e74b2 (127.18.118.129:41675)
I20251023 08:01:00.354789 18906 webserver.cc:492] Webserver started at http://127.18.118.130:43005/ using document root <none> and password file <none>
I20251023 08:01:00.355269 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:01:00.355451 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:01:00.355687 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:01:00.356201 23102 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56286
I20251023 08:01:00.356722 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-1-root/instance:
uuid: "eace648c7acd4fefbd7c9394bb4f5afd"
format_stamp: "Formatted at 2025-10-23 08:01:00 on dist-test-slave-wkn7"
I20251023 08:01:00.360970 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.003s
I20251023 08:01:00.363972 23241 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:01:00.364730 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20251023 08:01:00.364996 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-1-root
uuid: "eace648c7acd4fefbd7c9394bb4f5afd"
format_stamp: "Formatted at 2025-10-23 08:01:00 on dist-test-slave-wkn7"
I20251023 08:01:00.365324 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:01:00.395637 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:01:00.396836 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:01:00.398317 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:01:00.400494 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:01:00.400689 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:01:00.400906 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:01:00.401058 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:01:00.439925 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.130:44217
I20251023 08:01:00.440025 23303 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.130:44217 every 8 connection(s)
I20251023 08:01:00.444950 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:01:00.451711 23307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:01:00.455931 23308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:01:00.458426 23304 heartbeater.cc:344] Connected to a master server at 127.18.118.190:33813
I20251023 08:01:00.459009 23304 heartbeater.cc:461] Registering TS with master...
I20251023 08:01:00.459872 23304 heartbeater.cc:507] Master 127.18.118.190:33813 requested a full tablet report, sending...
I20251023 08:01:00.460351 18906 server_base.cc:1047] running on GCE node
W20251023 08:01:00.461207 23310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:01:00.462116 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20251023 08:01:00.462184 23102 ts_manager.cc:194] Registered new tserver with Master: eace648c7acd4fefbd7c9394bb4f5afd (127.18.118.130:44217)
W20251023 08:01:00.462322 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:01:00.462716 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206460462696 us; error 0 us; skew 500 ppm
I20251023 08:01:00.463294 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:01:00.463696 23102 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56300
I20251023 08:01:00.466200 18906 webserver.cc:492] Webserver started at http://127.18.118.131:37983/ using document root <none> and password file <none>
I20251023 08:01:00.466722 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:01:00.466914 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:01:00.467212 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:01:00.468294 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-2-root/instance:
uuid: "2bed9caa88dd4aad85238223f56cebd5"
format_stamp: "Formatted at 2025-10-23 08:01:00 on dist-test-slave-wkn7"
I20251023 08:01:00.472846 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20251023 08:01:00.476302 23315 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:01:00.477090 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20251023 08:01:00.477433 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-2-root
uuid: "2bed9caa88dd4aad85238223f56cebd5"
format_stamp: "Formatted at 2025-10-23 08:01:00 on dist-test-slave-wkn7"
I20251023 08:01:00.477715 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:01:00.494571 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:01:00.495760 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:01:00.497224 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:01:00.499372 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:01:00.499576 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:01:00.499805 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:01:00.499959 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:01:00.539377 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.131:35523
I20251023 08:01:00.539449 23377 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.131:35523 every 8 connection(s)
I20251023 08:01:00.544301 18906 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251023 08:01:00.551678 23381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:01:00.553467 23382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251023 08:01:00.555058 23384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251023 08:01:00.555603 23378 heartbeater.cc:344] Connected to a master server at 127.18.118.190:33813
I20251023 08:01:00.555938 23378 heartbeater.cc:461] Registering TS with master...
I20251023 08:01:00.556231 18906 server_base.cc:1047] running on GCE node
I20251023 08:01:00.556929 23378 heartbeater.cc:507] Master 127.18.118.190:33813 requested a full tablet report, sending...
I20251023 08:01:00.557303 18906 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251023 08:01:00.557591 18906 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251023 08:01:00.557811 18906 hybrid_clock.cc:648] HybridClock initialized: now 1761206460557791 us; error 0 us; skew 500 ppm
I20251023 08:01:00.558576 18906 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251023 08:01:00.559664 23102 ts_manager.cc:194] Registered new tserver with Master: 2bed9caa88dd4aad85238223f56cebd5 (127.18.118.131:35523)
I20251023 08:01:00.561467 23102 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56310
I20251023 08:01:00.561733 18906 webserver.cc:492] Webserver started at http://127.18.118.132:44763/ using document root <none> and password file <none>
I20251023 08:01:00.562299 18906 fs_manager.cc:362] Metadata directory not provided
I20251023 08:01:00.562593 18906 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251023 08:01:00.562853 18906 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251023 08:01:00.563915 18906 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-3-root/instance:
uuid: "d1bbe1092821462488d3aaaee13daf5b"
format_stamp: "Formatted at 2025-10-23 08:01:00 on dist-test-slave-wkn7"
I20251023 08:01:00.568533 18906 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20251023 08:01:00.572001 23389 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:01:00.572897 18906 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20251023 08:01:00.573256 18906 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-3-root
uuid: "d1bbe1092821462488d3aaaee13daf5b"
format_stamp: "Formatted at 2025-10-23 08:01:00 on dist-test-slave-wkn7"
I20251023 08:01:00.573531 18906 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251023 08:01:00.585593 18906 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20251023 08:01:00.586788 18906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251023 08:01:00.588248 18906 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251023 08:01:00.590492 18906 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251023 08:01:00.590677 18906 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:01:00.590919 18906 ts_tablet_manager.cc:616] Registered 0 tablets
I20251023 08:01:00.591064 18906 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20251023 08:01:00.630883 23451 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.18.118.132:39205 every 8 connection(s)
I20251023 08:01:00.630914 18906 rpc_server.cc:307] RPC server started. Bound to: 127.18.118.132:39205
I20251023 08:01:00.645509 23452 heartbeater.cc:344] Connected to a master server at 127.18.118.190:33813
I20251023 08:01:00.645939 23452 heartbeater.cc:461] Registering TS with master...
I20251023 08:01:00.646648 23452 heartbeater.cc:507] Master 127.18.118.190:33813 requested a full tablet report, sending...
I20251023 08:01:00.648609 23102 ts_manager.cc:194] Registered new tserver with Master: d1bbe1092821462488d3aaaee13daf5b (127.18.118.132:39205)
I20251023 08:01:00.649708 18906 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.015149591s
I20251023 08:01:00.650123 23102 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56320
I20251023 08:01:00.669046 18906 test_util.cc:276] Using random seed: -713630625
I20251023 08:01:00.691478 23102 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56342:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20251023 08:01:00.728618 23343 tablet_service.cc:1505] Processing CreateTablet for tablet 7e5af8e5141d47c782b96b7b8ec04c5e (DEFAULT_TABLE table=test-workload [id=0b2f60206df74a78bbc4cdbeadc9a1a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 08:01:00.729970 23343 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e5af8e5141d47c782b96b7b8ec04c5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:01:00.736750 23417 tablet_service.cc:1505] Processing CreateTablet for tablet 7e5af8e5141d47c782b96b7b8ec04c5e (DEFAULT_TABLE table=test-workload [id=0b2f60206df74a78bbc4cdbeadc9a1a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 08:01:00.738155 23417 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e5af8e5141d47c782b96b7b8ec04c5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:01:00.738867 23194 tablet_service.cc:1505] Processing CreateTablet for tablet 7e5af8e5141d47c782b96b7b8ec04c5e (DEFAULT_TABLE table=test-workload [id=0b2f60206df74a78bbc4cdbeadc9a1a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20251023 08:01:00.740069 23194 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e5af8e5141d47c782b96b7b8ec04c5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:01:00.750263 23472 tablet_bootstrap.cc:492] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5: Bootstrap starting.
I20251023 08:01:00.754920 23473 tablet_bootstrap.cc:492] T 7e5af8e5141d47c782b96b7b8ec04c5e P d1bbe1092821462488d3aaaee13daf5b: Bootstrap starting.
I20251023 08:01:00.758342 23472 tablet_bootstrap.cc:654] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5: Neither blocks nor log segments found. Creating new log.
I20251023 08:01:00.759151 23474 tablet_bootstrap.cc:492] T 7e5af8e5141d47c782b96b7b8ec04c5e P 6c26960f9a354d648d46267ff91e74b2: Bootstrap starting.
I20251023 08:01:00.763906 23473 tablet_bootstrap.cc:654] T 7e5af8e5141d47c782b96b7b8ec04c5e P d1bbe1092821462488d3aaaee13daf5b: Neither blocks nor log segments found. Creating new log.
I20251023 08:01:00.765317 23472 tablet_bootstrap.cc:492] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5: No bootstrap required, opened a new log
I20251023 08:01:00.765719 23472 ts_tablet_manager.cc:1403] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.005s
I20251023 08:01:00.767266 23474 tablet_bootstrap.cc:654] T 7e5af8e5141d47c782b96b7b8ec04c5e P 6c26960f9a354d648d46267ff91e74b2: Neither blocks nor log segments found. Creating new log.
I20251023 08:01:00.768409 23472 raft_consensus.cc:359] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bed9caa88dd4aad85238223f56cebd5" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 35523 } } peers { permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 } } peers { permanent_uuid: "d1bbe1092821462488d3aaaee13daf5b" member_type: VOTER last_known_addr { host: "127.18.118.132" port: 39205 } }
I20251023 08:01:00.768985 23472 raft_consensus.cc:385] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:01:00.769412 23472 raft_consensus.cc:740] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bed9caa88dd4aad85238223f56cebd5, State: Initialized, Role: FOLLOWER
I20251023 08:01:00.770100 23473 tablet_bootstrap.cc:492] T 7e5af8e5141d47c782b96b7b8ec04c5e P d1bbe1092821462488d3aaaee13daf5b: No bootstrap required, opened a new log
I20251023 08:01:00.770160 23472 consensus_queue.cc:260] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bed9caa88dd4aad85238223f56cebd5" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 35523 } } peers { permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 } } peers { permanent_uuid: "d1bbe1092821462488d3aaaee13daf5b" member_type: VOTER last_known_addr { host: "127.18.118.132" port: 39205 } }
I20251023 08:01:00.770906 23473 ts_tablet_manager.cc:1403] T 7e5af8e5141d47c782b96b7b8ec04c5e P d1bbe1092821462488d3aaaee13daf5b: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.005s
I20251023 08:01:00.773777 23474 tablet_bootstrap.cc:492] T 7e5af8e5141d47c782b96b7b8ec04c5e P 6c26960f9a354d648d46267ff91e74b2: No bootstrap required, opened a new log
I20251023 08:01:00.774271 23378 heartbeater.cc:499] Master 127.18.118.190:33813 was elected leader, sending a full tablet report...
I20251023 08:01:00.774338 23474 ts_tablet_manager.cc:1403] T 7e5af8e5141d47c782b96b7b8ec04c5e P 6c26960f9a354d648d46267ff91e74b2: Time spent bootstrapping tablet: real 0.015s	user 0.015s	sys 0.000s
I20251023 08:01:00.774910 23472 ts_tablet_manager.cc:1434] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.002s
I20251023 08:01:00.774801 23473 raft_consensus.cc:359] T 7e5af8e5141d47c782b96b7b8ec04c5e P d1bbe1092821462488d3aaaee13daf5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bed9caa88dd4aad85238223f56cebd5" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 35523 } } peers { permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 } } peers { permanent_uuid: "d1bbe1092821462488d3aaaee13daf5b" member_type: VOTER last_known_addr { host: "127.18.118.132" port: 39205 } }
I20251023 08:01:00.775832 23473 raft_consensus.cc:385] T 7e5af8e5141d47c782b96b7b8ec04c5e P d1bbe1092821462488d3aaaee13daf5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:01:00.776098 23473 raft_consensus.cc:740] T 7e5af8e5141d47c782b96b7b8ec04c5e P d1bbe1092821462488d3aaaee13daf5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1bbe1092821462488d3aaaee13daf5b, State: Initialized, Role: FOLLOWER
I20251023 08:01:00.777884 23474 raft_consensus.cc:359] T 7e5af8e5141d47c782b96b7b8ec04c5e P 6c26960f9a354d648d46267ff91e74b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bed9caa88dd4aad85238223f56cebd5" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 35523 } } peers { permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 } } peers { permanent_uuid: "d1bbe1092821462488d3aaaee13daf5b" member_type: VOTER last_known_addr { host: "127.18.118.132" port: 39205 } }
I20251023 08:01:00.778439 23474 raft_consensus.cc:385] T 7e5af8e5141d47c782b96b7b8ec04c5e P 6c26960f9a354d648d46267ff91e74b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251023 08:01:00.778659 23474 raft_consensus.cc:740] T 7e5af8e5141d47c782b96b7b8ec04c5e P 6c26960f9a354d648d46267ff91e74b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c26960f9a354d648d46267ff91e74b2, State: Initialized, Role: FOLLOWER
I20251023 08:01:00.779340 23474 consensus_queue.cc:260] T 7e5af8e5141d47c782b96b7b8ec04c5e P 6c26960f9a354d648d46267ff91e74b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bed9caa88dd4aad85238223f56cebd5" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 35523 } } peers { permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 } } peers { permanent_uuid: "d1bbe1092821462488d3aaaee13daf5b" member_type: VOTER last_known_addr { host: "127.18.118.132" port: 39205 } }
I20251023 08:01:00.779465 23473 consensus_queue.cc:260] T 7e5af8e5141d47c782b96b7b8ec04c5e P d1bbe1092821462488d3aaaee13daf5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bed9caa88dd4aad85238223f56cebd5" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 35523 } } peers { permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 } } peers { permanent_uuid: "d1bbe1092821462488d3aaaee13daf5b" member_type: VOTER last_known_addr { host: "127.18.118.132" port: 39205 } }
I20251023 08:01:00.782043 23474 ts_tablet_manager.cc:1434] T 7e5af8e5141d47c782b96b7b8ec04c5e P 6c26960f9a354d648d46267ff91e74b2: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20251023 08:01:00.781836 23229 heartbeater.cc:499] Master 127.18.118.190:33813 was elected leader, sending a full tablet report...
I20251023 08:01:00.788949 23452 heartbeater.cc:499] Master 127.18.118.190:33813 was elected leader, sending a full tablet report...
I20251023 08:01:00.790087 23473 ts_tablet_manager.cc:1434] T 7e5af8e5141d47c782b96b7b8ec04c5e P d1bbe1092821462488d3aaaee13daf5b: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.004s
I20251023 08:01:00.871752 23478 raft_consensus.cc:493] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251023 08:01:00.872227 23478 raft_consensus.cc:515] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bed9caa88dd4aad85238223f56cebd5" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 35523 } } peers { permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 } } peers { permanent_uuid: "d1bbe1092821462488d3aaaee13daf5b" member_type: VOTER last_known_addr { host: "127.18.118.132" port: 39205 } }
I20251023 08:01:00.874297 23478 leader_election.cc:290] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c26960f9a354d648d46267ff91e74b2 (127.18.118.129:41675), d1bbe1092821462488d3aaaee13daf5b (127.18.118.132:39205)
I20251023 08:01:00.884017 23204 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7e5af8e5141d47c782b96b7b8ec04c5e" candidate_uuid: "2bed9caa88dd4aad85238223f56cebd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c26960f9a354d648d46267ff91e74b2" is_pre_election: true
I20251023 08:01:00.884697 23204 raft_consensus.cc:2468] T 7e5af8e5141d47c782b96b7b8ec04c5e P 6c26960f9a354d648d46267ff91e74b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bed9caa88dd4aad85238223f56cebd5 in term 0.
I20251023 08:01:00.884872 23427 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7e5af8e5141d47c782b96b7b8ec04c5e" candidate_uuid: "2bed9caa88dd4aad85238223f56cebd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bbe1092821462488d3aaaee13daf5b" is_pre_election: true
I20251023 08:01:00.885568 23427 raft_consensus.cc:2468] T 7e5af8e5141d47c782b96b7b8ec04c5e P d1bbe1092821462488d3aaaee13daf5b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bed9caa88dd4aad85238223f56cebd5 in term 0.
I20251023 08:01:00.885946 23318 leader_election.cc:304] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bed9caa88dd4aad85238223f56cebd5, 6c26960f9a354d648d46267ff91e74b2; no voters: 
I20251023 08:01:00.886752 23478 raft_consensus.cc:2804] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251023 08:01:00.887042 23478 raft_consensus.cc:493] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251023 08:01:00.887296 23478 raft_consensus.cc:3060] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:01:00.892318 23478 raft_consensus.cc:515] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bed9caa88dd4aad85238223f56cebd5" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 35523 } } peers { permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 } } peers { permanent_uuid: "d1bbe1092821462488d3aaaee13daf5b" member_type: VOTER last_known_addr { host: "127.18.118.132" port: 39205 } }
I20251023 08:01:00.893975 23478 leader_election.cc:290] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [CANDIDATE]: Term 1 election: Requested vote from peers 6c26960f9a354d648d46267ff91e74b2 (127.18.118.129:41675), d1bbe1092821462488d3aaaee13daf5b (127.18.118.132:39205)
I20251023 08:01:00.895004 23427 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7e5af8e5141d47c782b96b7b8ec04c5e" candidate_uuid: "2bed9caa88dd4aad85238223f56cebd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bbe1092821462488d3aaaee13daf5b"
I20251023 08:01:00.895385 23204 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7e5af8e5141d47c782b96b7b8ec04c5e" candidate_uuid: "2bed9caa88dd4aad85238223f56cebd5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c26960f9a354d648d46267ff91e74b2"
I20251023 08:01:00.895485 23427 raft_consensus.cc:3060] T 7e5af8e5141d47c782b96b7b8ec04c5e P d1bbe1092821462488d3aaaee13daf5b [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:01:00.895884 23204 raft_consensus.cc:3060] T 7e5af8e5141d47c782b96b7b8ec04c5e P 6c26960f9a354d648d46267ff91e74b2 [term 0 FOLLOWER]: Advancing to term 1
I20251023 08:01:00.900328 23427 raft_consensus.cc:2468] T 7e5af8e5141d47c782b96b7b8ec04c5e P d1bbe1092821462488d3aaaee13daf5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bed9caa88dd4aad85238223f56cebd5 in term 1.
I20251023 08:01:00.900424 23204 raft_consensus.cc:2468] T 7e5af8e5141d47c782b96b7b8ec04c5e P 6c26960f9a354d648d46267ff91e74b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bed9caa88dd4aad85238223f56cebd5 in term 1.
I20251023 08:01:00.901407 23319 leader_election.cc:304] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bed9caa88dd4aad85238223f56cebd5, d1bbe1092821462488d3aaaee13daf5b; no voters: 
I20251023 08:01:00.902099 23478 raft_consensus.cc:2804] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [term 1 FOLLOWER]: Leader election won for term 1
I20251023 08:01:00.902931 23478 raft_consensus.cc:697] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [term 1 LEADER]: Becoming Leader. State: Replica: 2bed9caa88dd4aad85238223f56cebd5, State: Running, Role: LEADER
I20251023 08:01:00.903600 23478 consensus_queue.cc:237] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bed9caa88dd4aad85238223f56cebd5" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 35523 } } peers { permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 } } peers { permanent_uuid: "d1bbe1092821462488d3aaaee13daf5b" member_type: VOTER last_known_addr { host: "127.18.118.132" port: 39205 } }
I20251023 08:01:00.910043 23102 catalog_manager.cc:5649] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2bed9caa88dd4aad85238223f56cebd5 (127.18.118.131). New cstate: current_term: 1 leader_uuid: "2bed9caa88dd4aad85238223f56cebd5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bed9caa88dd4aad85238223f56cebd5" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 35523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1bbe1092821462488d3aaaee13daf5b" member_type: VOTER last_known_addr { host: "127.18.118.132" port: 39205 } health_report { overall_health: UNKNOWN } } }
I20251023 08:01:01.166018 23343 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251023 08:01:01.168304 23194 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:01:01.169864 23417 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:01:01.180140 23269 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:01:01.425134 23513 consensus_queue.cc:1048] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1bbe1092821462488d3aaaee13daf5b" member_type: VOTER last_known_addr { host: "127.18.118.132" port: 39205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251023 08:01:01.441846 23516 consensus_queue.cc:1048] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Master Summary
               UUID               |       Address        | Status
----------------------------------+----------------------+---------
 5fe8f347ed5541819166014f37970118 | 127.18.118.190:33813 | HEALTHY

Unusual flags for Master:
               Flag               | Value |  Tags  |         Master
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |       Address        | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
 2bed9caa88dd4aad85238223f56cebd5 | 127.18.118.131:35523 | HEALTHY | <none>   |       1        |       0
 6c26960f9a354d648d46267ff91e74b2 | 127.18.118.129:41675 | HEALTHY | <none>   |       0        |       0
 d1bbe1092821462488d3aaaee13daf5b | 127.18.118.132:39205 | HEALTHY | <none>   |       0        |       0
 eace648c7acd4fefbd7c9394bb4f5afd | 127.18.118.130:44217 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       4

Unusual flags for Tablet Server:
               Flag               | Value |  Tags  |      Tablet Server
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 4 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
 time_source         | system_unsync                                               | all 4 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 5 server(s) checked

Tablet Summary
Tablet 7e5af8e5141d47c782b96b7b8ec04c5e of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
  2bed9caa88dd4aad85238223f56cebd5 (127.18.118.131:35523): RUNNING [LEADER]
  6c26960f9a354d648d46267ff91e74b2 (127.18.118.129:41675): RUNNING
  d1bbe1092821462488d3aaaee13daf5b (127.18.118.132:39205): RUNNING
All reported replicas are:
  A = 2bed9caa88dd4aad85238223f56cebd5
  B = 6c26960f9a354d648d46267ff91e74b2
  C = d1bbe1092821462488d3aaaee13daf5b
The consensus matrix is:
 Config source |   Replicas   | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
 master        | A*  B   C    |              |              | Yes
 A             | A*  B   C    | 1            | -1           | Yes
 B             | A   B   C    | 1            | -1           | Yes
 C             | A   B   C    | 1            | -1           | Yes

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |       Status       | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | CONSENSUS_MISMATCH | 1             | 0       | 0          | 0                | 1

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 0
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Tablet Replica Count Outliers
 Type  |               UUID               |         Host         | Replica Count
-------+----------------------------------+----------------------+---------------
 Small | eace648c7acd4fefbd7c9394bb4f5afd | 127.18.118.130:44217 | 0

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 4
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20251023 08:01:01.462059 18906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20251023 08:01:01.466253 23304 heartbeater.cc:499] Master 127.18.118.190:33813 was elected leader, sending a full tablet report...
I20251023 08:01:01.829573 23194 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:01:01.834338 23343 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251023 08:01:01.841797 23417 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:01:01.850688 23269 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
               UUID               |       Address        | Status
----------------------------------+----------------------+---------
 5fe8f347ed5541819166014f37970118 | 127.18.118.190:33813 | HEALTHY

Unusual flags for Master:
               Flag               | Value |  Tags  |         Master
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |       Address        | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
 2bed9caa88dd4aad85238223f56cebd5 | 127.18.118.131:35523 | HEALTHY | <none>   |       1        |       0
 6c26960f9a354d648d46267ff91e74b2 | 127.18.118.129:41675 | HEALTHY | <none>   |       0        |       0
 d1bbe1092821462488d3aaaee13daf5b | 127.18.118.132:39205 | HEALTHY | <none>   |       0        |       0
 eace648c7acd4fefbd7c9394bb4f5afd | 127.18.118.130:44217 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       4

Unusual flags for Tablet Server:
               Flag               | Value |  Tags  |      Tablet Server
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 4 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
 time_source         | system_unsync                                               | all 4 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 5 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
     Name      | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 0
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Tablet Replica Count Outliers
 Type  |               UUID               |         Host         | Replica Count
-------+----------------------------------+----------------------+---------------
 Small | eace648c7acd4fefbd7c9394bb4f5afd | 127.18.118.130:44217 | 0

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 4
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20251023 08:01:02.107849 18906 log_verifier.cc:126] Checking tablet 7e5af8e5141d47c782b96b7b8ec04c5e
I20251023 08:01:02.166333 18906 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 7e5af8e5141d47c782b96b7b8ec04c5e
I20251023 08:01:02.166821 18906 tablet_replica.cc:333] T 7e5af8e5141d47c782b96b7b8ec04c5e P d1bbe1092821462488d3aaaee13daf5b: stopping tablet replica
I20251023 08:01:02.167295 18906 raft_consensus.cc:2243] T 7e5af8e5141d47c782b96b7b8ec04c5e P d1bbe1092821462488d3aaaee13daf5b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 08:01:02.167775 18906 raft_consensus.cc:2272] T 7e5af8e5141d47c782b96b7b8ec04c5e P d1bbe1092821462488d3aaaee13daf5b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:01:02.339751 23319 consensus_queue.cc:579] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [LEADER]: The tablet replica hosted on peer d1bbe1092821462488d3aaaee13daf5b has failed
W20251023 08:01:02.340211 23319 consensus_peers.cc:597] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 -> Peer d1bbe1092821462488d3aaaee13daf5b (127.18.118.132:39205): Couldn't send request to peer d1bbe1092821462488d3aaaee13daf5b. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20251023 08:01:02.347545 23353 consensus_queue.cc:237] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2bed9caa88dd4aad85238223f56cebd5" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 35523 } } peers { permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 } }
I20251023 08:01:02.351192 23204 raft_consensus.cc:1275] T 7e5af8e5141d47c782b96b7b8ec04c5e P 6c26960f9a354d648d46267ff91e74b2 [term 1 FOLLOWER]: Refusing update from remote peer 2bed9caa88dd4aad85238223f56cebd5: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251023 08:01:02.352511 23513 consensus_queue.cc:1048] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20251023 08:01:02.360450 23516 raft_consensus.cc:2955] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER d1bbe1092821462488d3aaaee13daf5b (127.18.118.132) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2bed9caa88dd4aad85238223f56cebd5" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 35523 } } peers { permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 } } }
I20251023 08:01:02.379884 23089 catalog_manager.cc:5162] ChangeConfig:REMOVE_PEER RPC for tablet 7e5af8e5141d47c782b96b7b8ec04c5e with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20251023 08:01:02.383631 23204 raft_consensus.cc:2955] T 7e5af8e5141d47c782b96b7b8ec04c5e P 6c26960f9a354d648d46267ff91e74b2 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER d1bbe1092821462488d3aaaee13daf5b (127.18.118.132) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2bed9caa88dd4aad85238223f56cebd5" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 35523 } } peers { permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 } } }
I20251023 08:01:02.385197 23101 catalog_manager.cc:5649] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 reported cstate change: config changed from index -1 to 2, VOTER d1bbe1092821462488d3aaaee13daf5b (127.18.118.132) evicted. New cstate: current_term: 1 leader_uuid: "2bed9caa88dd4aad85238223f56cebd5" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "2bed9caa88dd4aad85238223f56cebd5" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 35523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 } health_report { overall_health: HEALTHY } } }
I20251023 08:01:02.393723 23343 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251023 08:01:02.421900 23194 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:01:02.422533 23269 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:01:02.432313 23417 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:01:02.456669 23353 consensus_queue.cc:237] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2bed9caa88dd4aad85238223f56cebd5" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 35523 } } peers { permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 } } peers { permanent_uuid: "eace648c7acd4fefbd7c9394bb4f5afd" member_type: NON_VOTER last_known_addr { host: "127.18.118.130" port: 44217 } attrs { promote: true } }
I20251023 08:01:02.461349 23417 tablet_service.cc:1552] Processing DeleteTablet for tablet 7e5af8e5141d47c782b96b7b8ec04c5e with delete_type TABLET_DATA_TOMBSTONED (TS d1bbe1092821462488d3aaaee13daf5b not found in new config with opid_index 2) from {username='slave'} at 127.0.0.1:52884
I20251023 08:01:02.476897 23585 ts_tablet_manager.cc:1916] T 7e5af8e5141d47c782b96b7b8ec04c5e P d1bbe1092821462488d3aaaee13daf5b: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20251023 08:01:02.493274 23585 ts_tablet_manager.cc:1929] T 7e5af8e5141d47c782b96b7b8ec04c5e P d1bbe1092821462488d3aaaee13daf5b: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20251023 08:01:02.493877 23585 log.cc:1199] T 7e5af8e5141d47c782b96b7b8ec04c5e P d1bbe1092821462488d3aaaee13daf5b: Deleting WAL directory at /tmp/dist-test-taskFqnw2O/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1761206396810293-18906-0/minicluster-data/ts-3-root/wals/7e5af8e5141d47c782b96b7b8ec04c5e
I20251023 08:01:02.495674 23089 catalog_manager.cc:4985] TS d1bbe1092821462488d3aaaee13daf5b (127.18.118.132:39205): tablet 7e5af8e5141d47c782b96b7b8ec04c5e (table test-workload [id=0b2f60206df74a78bbc4cdbeadc9a1a9]) successfully deleted
I20251023 08:01:02.525648 23204 raft_consensus.cc:1275] T 7e5af8e5141d47c782b96b7b8ec04c5e P 6c26960f9a354d648d46267ff91e74b2 [term 1 FOLLOWER]: Refusing update from remote peer 2bed9caa88dd4aad85238223f56cebd5: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20251023 08:01:02.527336 23516 consensus_queue.cc:1048] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251023 08:01:02.534870 23516 raft_consensus.cc:2955] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER eace648c7acd4fefbd7c9394bb4f5afd (127.18.118.130) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2bed9caa88dd4aad85238223f56cebd5" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 35523 } } peers { permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 } } peers { permanent_uuid: "eace648c7acd4fefbd7c9394bb4f5afd" member_type: NON_VOTER last_known_addr { host: "127.18.118.130" port: 44217 } attrs { promote: true } } }
I20251023 08:01:02.538417 23204 raft_consensus.cc:2955] T 7e5af8e5141d47c782b96b7b8ec04c5e P 6c26960f9a354d648d46267ff91e74b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER eace648c7acd4fefbd7c9394bb4f5afd (127.18.118.130) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2bed9caa88dd4aad85238223f56cebd5" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 35523 } } peers { permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 } } peers { permanent_uuid: "eace648c7acd4fefbd7c9394bb4f5afd" member_type: NON_VOTER last_known_addr { host: "127.18.118.130" port: 44217 } attrs { promote: true } } }
I20251023 08:01:02.562175 23089 catalog_manager.cc:5162] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 7e5af8e5141d47c782b96b7b8ec04c5e with cas_config_opid_index 2: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251023 08:01:02.567705 23100 catalog_manager.cc:5649] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 reported cstate change: config changed from index 2 to 3, NON_VOTER eace648c7acd4fefbd7c9394bb4f5afd (127.18.118.130) added. New cstate: current_term: 1 leader_uuid: "2bed9caa88dd4aad85238223f56cebd5" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2bed9caa88dd4aad85238223f56cebd5" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 35523 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eace648c7acd4fefbd7c9394bb4f5afd" member_type: NON_VOTER last_known_addr { host: "127.18.118.130" port: 44217 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20251023 08:01:02.583956 23318 consensus_peers.cc:597] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 -> Peer eace648c7acd4fefbd7c9394bb4f5afd (127.18.118.130:44217): Couldn't send request to peer eace648c7acd4fefbd7c9394bb4f5afd. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7e5af8e5141d47c782b96b7b8ec04c5e. This is attempt 1: this message will repeat every 5th retry.
Master Summary
               UUID               |       Address        | Status
----------------------------------+----------------------+---------
 5fe8f347ed5541819166014f37970118 | 127.18.118.190:33813 | HEALTHY

Unusual flags for Master:
               Flag               | Value |  Tags  |         Master
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |       Address        | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
 2bed9caa88dd4aad85238223f56cebd5 | 127.18.118.131:35523 | HEALTHY | <none>   |       1        |       0
 6c26960f9a354d648d46267ff91e74b2 | 127.18.118.129:41675 | HEALTHY | <none>   |       0        |       0
 d1bbe1092821462488d3aaaee13daf5b | 127.18.118.132:39205 | HEALTHY | <none>   |       0        |       0
 eace648c7acd4fefbd7c9394bb4f5afd | 127.18.118.130:44217 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       4

Unusual flags for Tablet Server:
               Flag               | Value |  Tags  |      Tablet Server
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 4 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
 time_source         | system_unsync                                               | all 4 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 5 server(s) checked

Tablet Summary
Tablet 7e5af8e5141d47c782b96b7b8ec04c5e of table 'test-workload' is under-replicated: 1 replica(s) not RUNNING
  2bed9caa88dd4aad85238223f56cebd5 (127.18.118.131:35523): RUNNING [LEADER]
  6c26960f9a354d648d46267ff91e74b2 (127.18.118.129:41675): RUNNING
  d1bbe1092821462488d3aaaee13daf5b (127.18.118.132:39205): not running
    State:       FAILED
    Data state:  TABLET_DATA_READY
    Last status: IO error: INJECTED ERROR: tablet failed
All reported replicas are:
  A = 2bed9caa88dd4aad85238223f56cebd5
  B = 6c26960f9a354d648d46267ff91e74b2
  C = d1bbe1092821462488d3aaaee13daf5b
The consensus matrix is:
 Config source |        Replicas        | Current term | Config index | Committed?
---------------+------------------------+--------------+--------------+------------
 master        | A*  B   C              |              |              | Yes
 A             | A*  B                  | 1            | 2            | Yes
 B             | A*  B                  | 1            | 2            | Yes
 C             | [config not available] |              |              | 

The cluster doesn't have any matching system tables
Summary by table
     Name      | RF |      Status      | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+------------------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | UNDER_REPLICATED | 1             | 0       | 0          | 1                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 0
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Tablet Replica Count Outliers
 Type  |               UUID               |         Host         | Replica Count
-------+----------------------------------+----------------------+---------------
 Small | eace648c7acd4fefbd7c9394bb4f5afd | 127.18.118.130:44217 | 0

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 4
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy

FAILED
I20251023 08:01:02.696658 18906 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20251023 08:01:03.074432 23194 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:01:03.074460 23343 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251023 08:01:03.077081 23417 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:01:03.081947 23269 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251023 08:01:03.135852 23616 ts_tablet_manager.cc:933] T 7e5af8e5141d47c782b96b7b8ec04c5e P eace648c7acd4fefbd7c9394bb4f5afd: Initiating tablet copy from peer 2bed9caa88dd4aad85238223f56cebd5 (127.18.118.131:35523)
I20251023 08:01:03.161254 23616 tablet_copy_client.cc:323] T 7e5af8e5141d47c782b96b7b8ec04c5e P eace648c7acd4fefbd7c9394bb4f5afd: tablet copy: Beginning tablet copy session from remote peer at address 127.18.118.131:35523
I20251023 08:01:03.198801 23363 tablet_copy_service.cc:140] P 2bed9caa88dd4aad85238223f56cebd5: Received BeginTabletCopySession request for tablet 7e5af8e5141d47c782b96b7b8ec04c5e from peer eace648c7acd4fefbd7c9394bb4f5afd ({username='slave'} at 127.0.0.1:53808)
I20251023 08:01:03.199417 23363 tablet_copy_service.cc:161] P 2bed9caa88dd4aad85238223f56cebd5: Beginning new tablet copy session on tablet 7e5af8e5141d47c782b96b7b8ec04c5e from peer eace648c7acd4fefbd7c9394bb4f5afd at {username='slave'} at 127.0.0.1:53808: session id = eace648c7acd4fefbd7c9394bb4f5afd-7e5af8e5141d47c782b96b7b8ec04c5e
I20251023 08:01:03.207437 23363 tablet_copy_source_session.cc:215] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5: Tablet Copy: opened 0 blocks and 1 log segments
I20251023 08:01:03.210745 23616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e5af8e5141d47c782b96b7b8ec04c5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251023 08:01:03.241470 23616 tablet_copy_client.cc:806] T 7e5af8e5141d47c782b96b7b8ec04c5e P eace648c7acd4fefbd7c9394bb4f5afd: tablet copy: Starting download of 0 data blocks...
I20251023 08:01:03.242092 23616 tablet_copy_client.cc:670] T 7e5af8e5141d47c782b96b7b8ec04c5e P eace648c7acd4fefbd7c9394bb4f5afd: tablet copy: Starting download of 1 WAL segments...
I20251023 08:01:03.246510 23616 tablet_copy_client.cc:538] T 7e5af8e5141d47c782b96b7b8ec04c5e P eace648c7acd4fefbd7c9394bb4f5afd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251023 08:01:03.256321 23616 tablet_bootstrap.cc:492] T 7e5af8e5141d47c782b96b7b8ec04c5e P eace648c7acd4fefbd7c9394bb4f5afd: Bootstrap starting.
I20251023 08:01:03.283967 23616 tablet_bootstrap.cc:492] T 7e5af8e5141d47c782b96b7b8ec04c5e P eace648c7acd4fefbd7c9394bb4f5afd: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251023 08:01:03.284900 23616 tablet_bootstrap.cc:492] T 7e5af8e5141d47c782b96b7b8ec04c5e P eace648c7acd4fefbd7c9394bb4f5afd: Bootstrap complete.
I20251023 08:01:03.285696 23616 ts_tablet_manager.cc:1403] T 7e5af8e5141d47c782b96b7b8ec04c5e P eace648c7acd4fefbd7c9394bb4f5afd: Time spent bootstrapping tablet: real 0.030s	user 0.022s	sys 0.008s
I20251023 08:01:03.288849 23616 raft_consensus.cc:359] T 7e5af8e5141d47c782b96b7b8ec04c5e P eace648c7acd4fefbd7c9394bb4f5afd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2bed9caa88dd4aad85238223f56cebd5" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 35523 } } peers { permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 } } peers { permanent_uuid: "eace648c7acd4fefbd7c9394bb4f5afd" member_type: NON_VOTER last_known_addr { host: "127.18.118.130" port: 44217 } attrs { promote: true } }
I20251023 08:01:03.289781 23616 raft_consensus.cc:740] T 7e5af8e5141d47c782b96b7b8ec04c5e P eace648c7acd4fefbd7c9394bb4f5afd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: eace648c7acd4fefbd7c9394bb4f5afd, State: Initialized, Role: LEARNER
I20251023 08:01:03.290513 23616 consensus_queue.cc:260] T 7e5af8e5141d47c782b96b7b8ec04c5e P eace648c7acd4fefbd7c9394bb4f5afd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "2bed9caa88dd4aad85238223f56cebd5" member_type: VOTER last_known_addr { host: "127.18.118.131" port: 35523 } } peers { permanent_uuid: "6c26960f9a354d648d46267ff91e74b2" member_type: VOTER last_known_addr { host: "127.18.118.129" port: 41675 } } peers { permanent_uuid: "eace648c7acd4fefbd7c9394bb4f5afd" member_type: NON_VOTER last_known_addr { host: "127.18.118.130" port: 44217 } attrs { promote: true } }
I20251023 08:01:03.300057 23616 ts_tablet_manager.cc:1434] T 7e5af8e5141d47c782b96b7b8ec04c5e P eace648c7acd4fefbd7c9394bb4f5afd: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.007s
I20251023 08:01:03.303258 23363 tablet_copy_service.cc:342] P 2bed9caa88dd4aad85238223f56cebd5: Request end of tablet copy session eace648c7acd4fefbd7c9394bb4f5afd-7e5af8e5141d47c782b96b7b8ec04c5e received from {username='slave'} at 127.0.0.1:53808
I20251023 08:01:03.303695 23363 tablet_copy_service.cc:434] P 2bed9caa88dd4aad85238223f56cebd5: ending tablet copy session eace648c7acd4fefbd7c9394bb4f5afd-7e5af8e5141d47c782b96b7b8ec04c5e on tablet 7e5af8e5141d47c782b96b7b8ec04c5e with peer eace648c7acd4fefbd7c9394bb4f5afd
Master Summary
               UUID               |       Address        | Status
----------------------------------+----------------------+---------
 5fe8f347ed5541819166014f37970118 | 127.18.118.190:33813 | HEALTHY

Unusual flags for Master:
               Flag               | Value |  Tags  |         Master
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 1 server(s) checked

Flags of checked categories for Master:
        Flag         |                            Value                            |         Master
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
 time_source         | system_unsync                                               | all 1 server(s) checked

Tablet Server Summary
               UUID               |       Address        | Status  | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
 2bed9caa88dd4aad85238223f56cebd5 | 127.18.118.131:35523 | HEALTHY | <none>   |       1        |       0
 6c26960f9a354d648d46267ff91e74b2 | 127.18.118.129:41675 | HEALTHY | <none>   |       0        |       0
 d1bbe1092821462488d3aaaee13daf5b | 127.18.118.132:39205 | HEALTHY | <none>   |       0        |       0
 eace648c7acd4fefbd7c9394bb4f5afd | 127.18.118.130:44217 | HEALTHY | <none>   |       0        |       0

Tablet Server Location Summary
 Location |  Count
----------+---------
 <none>   |       4

Unusual flags for Tablet Server:
               Flag               | Value |  Tags  |      Tablet Server
----------------------------------+-------+--------+-------------------------
 rpc_server_allow_ephemeral_ports | true  | unsafe | all 4 server(s) checked

Flags of checked categories for Tablet Server:
        Flag         |                            Value                            |      Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
 builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
 time_source         | system_unsync                                               | all 4 server(s) checked

Version Summary
     Version     |         Servers
-----------------+-------------------------
 1.19.0-SNAPSHOT | all 5 server(s) checked

Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
     Name      | RF | Status  | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
 test-workload | 3  | HEALTHY | 1             | 1       | 0          | 0                | 0

Tablet Replica Count Summary
   Statistic    | Replica Count
----------------+---------------
 Minimum        | 0
 First Quartile | 1
 Median         | 1
 Third Quartile | 1
 Maximum        | 1

Tablet Replica Count Outliers
 Type  |               UUID               |         Host         | Replica Count
-------+----------------------------------+----------------------+---------------
 Small | d1bbe1092821462488d3aaaee13daf5b | 127.18.118.132:39205 | 0

Total Count Summary
                | Total Count
----------------+-------------
 Masters        | 1
 Tablet Servers | 4
 Tables         | 1
 Tablets        | 1
 Replicas       | 3

==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set

OK
I20251023 08:01:03.382853 18906 log_verifier.cc:126] Checking tablet 7e5af8e5141d47c782b96b7b8ec04c5e
I20251023 08:01:03.438514 18906 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 7e5af8e5141d47c782b96b7b8ec04c5e
I20251023 08:01:03.441151 18906 tablet_server.cc:178] TabletServer@127.18.118.129:0 shutting down...
I20251023 08:01:03.463325 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:01:03.463968 18906 tablet_replica.cc:333] T 7e5af8e5141d47c782b96b7b8ec04c5e P 6c26960f9a354d648d46267ff91e74b2: stopping tablet replica
I20251023 08:01:03.464563 18906 raft_consensus.cc:2243] T 7e5af8e5141d47c782b96b7b8ec04c5e P 6c26960f9a354d648d46267ff91e74b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251023 08:01:03.465242 18906 raft_consensus.cc:2272] T 7e5af8e5141d47c782b96b7b8ec04c5e P 6c26960f9a354d648d46267ff91e74b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:01:03.485312 18906 tablet_server.cc:195] TabletServer@127.18.118.129:0 shutdown complete.
I20251023 08:01:03.494086 18906 tablet_server.cc:178] TabletServer@127.18.118.130:0 shutting down...
W20251023 08:01:03.507053 23318 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.18.118.129:41675: connect: Connection refused (error 111) [suppressed 4 similar messages]
W20251023 08:01:03.513046 23318 consensus_peers.cc:597] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 -> Peer 6c26960f9a354d648d46267ff91e74b2 (127.18.118.129:41675): Couldn't send request to peer 6c26960f9a354d648d46267ff91e74b2. Status: Network error: Client connection negotiation failed: client connection to 127.18.118.129:41675: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251023 08:01:03.515024 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:01:03.515959 18906 tablet_replica.cc:333] T 7e5af8e5141d47c782b96b7b8ec04c5e P eace648c7acd4fefbd7c9394bb4f5afd: stopping tablet replica
I20251023 08:01:03.516520 18906 raft_consensus.cc:2243] T 7e5af8e5141d47c782b96b7b8ec04c5e P eace648c7acd4fefbd7c9394bb4f5afd [term 1 LEARNER]: Raft consensus shutting down.
I20251023 08:01:03.516914 18906 raft_consensus.cc:2272] T 7e5af8e5141d47c782b96b7b8ec04c5e P eace648c7acd4fefbd7c9394bb4f5afd [term 1 LEARNER]: Raft consensus is shut down!
I20251023 08:01:03.536906 18906 tablet_server.cc:195] TabletServer@127.18.118.130:0 shutdown complete.
I20251023 08:01:03.546211 18906 tablet_server.cc:178] TabletServer@127.18.118.131:0 shutting down...
I20251023 08:01:03.565014 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:01:03.565675 18906 tablet_replica.cc:333] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5: stopping tablet replica
I20251023 08:01:03.566243 18906 raft_consensus.cc:2243] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [term 1 LEADER]: Raft consensus shutting down.
I20251023 08:01:03.567095 18906 raft_consensus.cc:2272] T 7e5af8e5141d47c782b96b7b8ec04c5e P 2bed9caa88dd4aad85238223f56cebd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:01:03.586334 18906 tablet_server.cc:195] TabletServer@127.18.118.131:0 shutdown complete.
I20251023 08:01:03.595459 18906 tablet_server.cc:178] TabletServer@127.18.118.132:0 shutting down...
I20251023 08:01:03.613323 18906 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251023 08:01:03.629973 18906 tablet_server.cc:195] TabletServer@127.18.118.132:0 shutdown complete.
I20251023 08:01:03.638639 18906 master.cc:561] Master@127.18.118.190:33813 shutting down...
I20251023 08:01:03.652590 18906 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118 [term 1 LEADER]: Raft consensus shutting down.
I20251023 08:01:03.653075 18906 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251023 08:01:03.653384 18906 tablet_replica.cc:333] T 00000000000000000000000000000000 P 5fe8f347ed5541819166014f37970118: stopping tablet replica
I20251023 08:01:03.670797 18906 master.cc:583] Master@127.18.118.190:33813 shutdown complete.
[       OK ] FailedTabletsAreReplacedITest.OneReplica/1 (3659 ms)
[----------] 2 tests from FailedTabletsAreReplacedITest (10599 ms total)

[----------] Global test environment tear-down
[==========] 13 tests from 4 test suites ran. (66721 ms total)
[  PASSED  ] 11 tests.
[  SKIPPED ] 1 test, listed below:
[  SKIPPED ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange
[  FAILED  ] 1 test, listed below:
[  FAILED  ] TsTabletManagerITest.TestTableStats

 1 FAILED TEST
I20251023 08:01:03.705688 18906 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/meta_cache.cc:788: suppressed but not reported on 3 messages since previous log ~11 seconds ago
I20251023 08:01:03.705963 18906 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:5101: suppressed but not reported on 2 messages since previous log ~46 seconds ago
I20251023 08:01:03.706133 18906 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 97 messages since previous log ~58 seconds ago
I20251023 08:01:03.706350 18906 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet.cc:2378: suppressed but not reported on 8 messages since previous log ~14 seconds ago