[==========] Running 13 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 3 tests from LeadershipChangeReportingTest
[ RUN ] LeadershipChangeReportingTest.TestConcurrentGetTableLocations
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 03:41:20.179646 16369 thread.cc:636] rpc reactor (reactor) Time spent creating pthread: real 1.337s user 0.480s sys 0.856s
W20260806 03:41:20.180662 16369 thread.cc:602] rpc reactor (reactor) Time spent starting thread: real 1.338s user 0.480s sys 0.856s
I20260806 03:41:20.185194 16369 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.252.126:46735
I20260806 03:41:20.188028 16369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:41:20.189762 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:20.213683 16381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:20.215662 16380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:20.219482 16384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:20.346366 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:20.347980 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:20.348423 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:20.348726 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987680348696 us; error 0 us; skew 500 ppm
I20260806 03:41:20.350734 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:20.363147 16369 webserver.cc:460] Webserver started at http://127.15.252.126:32819/ using document root <none> and password file <none>
I20260806 03:41:20.364506 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:20.364733 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:20.365422 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:20.375399 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1785987678583700-16369-0/minicluster-data/master-0-root/instance:
uuid: "53ca9e2083054640a7fc3f22a094277e"
format_stamp: "Formatted at 2026-08-06 03:41:20 on dist-test-slave-csg5"
I20260806 03:41:20.385874 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.009s sys 0.001s
I20260806 03:41:20.393347 16389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:20.395583 16369 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.000s
I20260806 03:41:20.396134 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1785987678583700-16369-0/minicluster-data/master-0-root
uuid: "53ca9e2083054640a7fc3f22a094277e"
format_stamp: "Formatted at 2026-08-06 03:41:20 on dist-test-slave-csg5"
I20260806 03:41:20.396567 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1785987678583700-16369-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1785987678583700-16369-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1785987678583700-16369-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:20.417841 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:20.419883 16369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:41:20.420490 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:20.510342 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.126:46735
I20260806 03:41:20.510432 16441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.126:46735 every 8 connection(s)
I20260806 03:41:20.520097 16442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:20.550060 16442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e: Bootstrap starting.
I20260806 03:41:20.559404 16442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:20.561987 16442 log.cc:824] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e: Log is configured to *not* fsync() on all Append() calls
I20260806 03:41:20.567981 16442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e: No bootstrap required, opened a new log
I20260806 03:41:20.594828 16442 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca9e2083054640a7fc3f22a094277e" member_type: VOTER }
I20260806 03:41:20.595626 16442 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:20.596093 16442 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ca9e2083054640a7fc3f22a094277e, State: Initialized, Role: FOLLOWER
I20260806 03:41:20.597501 16442 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca9e2083054640a7fc3f22a094277e" member_type: VOTER }
I20260806 03:41:20.598260 16442 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:41:20.598603 16442 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:41:20.599104 16442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:20.603922 16442 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca9e2083054640a7fc3f22a094277e" member_type: VOTER }
I20260806 03:41:20.605221 16442 leader_election.cc:302] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53ca9e2083054640a7fc3f22a094277e; no voters:
I20260806 03:41:20.607577 16442 leader_election.cc:288] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:41:20.608073 16445 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:20.611300 16445 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e [term 1 LEADER]: Becoming Leader. State: Replica: 53ca9e2083054640a7fc3f22a094277e, State: Running, Role: LEADER
I20260806 03:41:20.612591 16445 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca9e2083054640a7fc3f22a094277e" member_type: VOTER }
I20260806 03:41:20.615152 16442 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:41:20.625021 16447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53ca9e2083054640a7fc3f22a094277e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca9e2083054640a7fc3f22a094277e" member_type: VOTER } }
I20260806 03:41:20.624989 16446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53ca9e2083054640a7fc3f22a094277e. Latest consensus state: current_term: 1 leader_uuid: "53ca9e2083054640a7fc3f22a094277e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53ca9e2083054640a7fc3f22a094277e" member_type: VOTER } }
I20260806 03:41:20.626037 16447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:20.626067 16446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:20.632823 16454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:41:20.651051 16454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:41:20.653623 16369 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260806 03:41:20.673113 16454 catalog_manager.cc:1347] Generated new cluster ID: 384aeb892dff43979ea98335f43ff7a0
I20260806 03:41:20.673496 16454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:41:20.699832 16454 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:41:20.702961 16454 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:41:20.723906 16454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e: Generated new TSK 0
I20260806 03:41:20.725636 16454 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:41:20.785490 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:20.795861 16464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:20.797937 16465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:20.802640 16369 server_base.cc:1034] running on GCE node
W20260806 03:41:20.803030 16467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:20.803994 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:20.804190 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:20.804404 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987680804389 us; error 0 us; skew 500 ppm
I20260806 03:41:20.805074 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:20.810449 16369 webserver.cc:460] Webserver started at http://127.15.252.65:44077/ using document root <none> and password file <none>
I20260806 03:41:20.811128 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:20.811352 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:20.811666 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:20.813169 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1785987678583700-16369-0/minicluster-data/ts-0-root/instance:
uuid: "29aaff6107c840a78be5a79c7ce6bb1e"
format_stamp: "Formatted at 2026-08-06 03:41:20 on dist-test-slave-csg5"
I20260806 03:41:20.820179 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260806 03:41:20.825570 16472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:20.826553 16369 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260806 03:41:20.826993 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1785987678583700-16369-0/minicluster-data/ts-0-root
uuid: "29aaff6107c840a78be5a79c7ce6bb1e"
format_stamp: "Formatted at 2026-08-06 03:41:20 on dist-test-slave-csg5"
I20260806 03:41:20.827430 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1785987678583700-16369-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1785987678583700-16369-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1785987678583700-16369-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:20.870282 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:20.872314 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:20.876892 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:20.884302 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:20.884824 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260806 03:41:20.885217 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:20.885459 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:21.000670 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.65:32915
I20260806 03:41:21.001530 16534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.65:32915 every 8 connection(s)
I20260806 03:41:21.012861 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:21.027254 16539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:21.032835 16540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:21.039350 16542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:21.040926 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:21.042061 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:21.042369 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:21.042580 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987681042566 us; error 0 us; skew 500 ppm
I20260806 03:41:21.043377 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:21.047259 16369 webserver.cc:460] Webserver started at http://127.15.252.66:43995/ using document root <none> and password file <none>
I20260806 03:41:21.048007 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:21.048269 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:21.048610 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:21.050180 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1785987678583700-16369-0/minicluster-data/ts-1-root/instance:
uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62"
format_stamp: "Formatted at 2026-08-06 03:41:21 on dist-test-slave-csg5"
I20260806 03:41:21.052680 16535 heartbeater.cc:347] Connected to a master server at 127.15.252.126:46735
I20260806 03:41:21.053164 16535 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:21.054301 16535 heartbeater.cc:511] Master 127.15.252.126:46735 requested a full tablet report, sending...
I20260806 03:41:21.058300 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260806 03:41:21.059659 16406 ts_manager.cc:194] Registered new tserver with Master: 29aaff6107c840a78be5a79c7ce6bb1e (127.15.252.65:32915)
I20260806 03:41:21.063865 16547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:21.064086 16406 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40864
I20260806 03:41:21.065016 16369 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260806 03:41:21.065412 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1785987678583700-16369-0/minicluster-data/ts-1-root
uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62"
format_stamp: "Formatted at 2026-08-06 03:41:21 on dist-test-slave-csg5"
I20260806 03:41:21.065853 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1785987678583700-16369-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1785987678583700-16369-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestConcurrentGetTableLocations.1785987678583700-16369-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:21.091148 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:21.092583 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:21.095146 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:21.099437 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:21.099745 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:21.100088 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:21.100301 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:21.170383 16535 heartbeater.cc:503] Master 127.15.252.126:46735 was elected leader, sending a full tablet report...
I20260806 03:41:21.213318 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.66:43771
I20260806 03:41:21.213562 16610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.66:43771 every 8 connection(s)
I20260806 03:41:21.236526 16611 heartbeater.cc:347] Connected to a master server at 127.15.252.126:46735
I20260806 03:41:21.237135 16611 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:21.238070 16611 heartbeater.cc:511] Master 127.15.252.126:46735 requested a full tablet report, sending...
I20260806 03:41:21.240439 16406 ts_manager.cc:194] Registered new tserver with Master: 41affeb1e1c24f0a93ec9d5a02fe4c62 (127.15.252.66:43771)
I20260806 03:41:21.240538 16369 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.020008712s
I20260806 03:41:21.242316 16406 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40880
I20260806 03:41:21.279770 16406 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40886:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:41:21.307334 16406 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20260806 03:41:21.346561 16611 heartbeater.cc:503] Master 127.15.252.126:46735 was elected leader, sending a full tablet report...
I20260806 03:41:21.382359 16500 tablet_service.cc:1467] Processing CreateTablet for tablet 6284bbe65e3348bb9f9e585b9af87197 (DEFAULT_TABLE table=test-table [id=d018a66023044fedac893505d4a849dd]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260806 03:41:21.384209 16500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6284bbe65e3348bb9f9e585b9af87197. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:21.387918 16499 tablet_service.cc:1467] Processing CreateTablet for tablet 0090c4be7d27488d87628d5297d54b72 (DEFAULT_TABLE table=test-table [id=d018a66023044fedac893505d4a849dd]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260806 03:41:21.389396 16499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0090c4be7d27488d87628d5297d54b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:21.389173 16576 tablet_service.cc:1467] Processing CreateTablet for tablet 6284bbe65e3348bb9f9e585b9af87197 (DEFAULT_TABLE table=test-table [id=d018a66023044fedac893505d4a849dd]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260806 03:41:21.390547 16576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6284bbe65e3348bb9f9e585b9af87197. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:21.396020 16575 tablet_service.cc:1467] Processing CreateTablet for tablet 0090c4be7d27488d87628d5297d54b72 (DEFAULT_TABLE table=test-table [id=d018a66023044fedac893505d4a849dd]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260806 03:41:21.397617 16575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0090c4be7d27488d87628d5297d54b72. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:21.412027 16627 tablet_bootstrap.cc:492] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e: Bootstrap starting.
I20260806 03:41:21.421814 16627 tablet_bootstrap.cc:654] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:21.430605 16628 tablet_bootstrap.cc:492] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62: Bootstrap starting.
I20260806 03:41:21.432377 16627 tablet_bootstrap.cc:492] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e: No bootstrap required, opened a new log
I20260806 03:41:21.432822 16627 ts_tablet_manager.cc:1397] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.012s
I20260806 03:41:21.435312 16627 raft_consensus.cc:348] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } }
I20260806 03:41:21.435909 16627 raft_consensus.cc:729] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29aaff6107c840a78be5a79c7ce6bb1e, State: Initialized, Role: FOLLOWER
I20260806 03:41:21.436427 16627 consensus_queue.cc:260] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } }
I20260806 03:41:21.440243 16627 ts_tablet_manager.cc:1428] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e: Time spent starting tablet: real 0.007s user 0.000s sys 0.008s
I20260806 03:41:21.440621 16628 tablet_bootstrap.cc:654] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:21.441615 16627 tablet_bootstrap.cc:492] T 0090c4be7d27488d87628d5297d54b72 P 29aaff6107c840a78be5a79c7ce6bb1e: Bootstrap starting.
I20260806 03:41:21.451045 16627 tablet_bootstrap.cc:654] T 0090c4be7d27488d87628d5297d54b72 P 29aaff6107c840a78be5a79c7ce6bb1e: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:21.454823 16628 tablet_bootstrap.cc:492] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62: No bootstrap required, opened a new log
I20260806 03:41:21.455379 16628 ts_tablet_manager.cc:1397] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62: Time spent bootstrapping tablet: real 0.025s user 0.010s sys 0.008s
I20260806 03:41:21.459739 16628 raft_consensus.cc:348] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } }
I20260806 03:41:21.460575 16628 raft_consensus.cc:729] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41affeb1e1c24f0a93ec9d5a02fe4c62, State: Initialized, Role: FOLLOWER
I20260806 03:41:21.461194 16628 consensus_queue.cc:260] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } }
I20260806 03:41:21.470896 16628 ts_tablet_manager.cc:1428] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62: Time spent starting tablet: real 0.015s user 0.008s sys 0.004s
W20260806 03:41:21.471454 16612 tablet.cc:2369] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:41:21.470798 16627 tablet_bootstrap.cc:492] T 0090c4be7d27488d87628d5297d54b72 P 29aaff6107c840a78be5a79c7ce6bb1e: No bootstrap required, opened a new log
I20260806 03:41:21.472491 16627 ts_tablet_manager.cc:1397] T 0090c4be7d27488d87628d5297d54b72 P 29aaff6107c840a78be5a79c7ce6bb1e: Time spent bootstrapping tablet: real 0.031s user 0.016s sys 0.009s
I20260806 03:41:21.473536 16628 tablet_bootstrap.cc:492] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62: Bootstrap starting.
I20260806 03:41:21.476722 16627 raft_consensus.cc:348] T 0090c4be7d27488d87628d5297d54b72 P 29aaff6107c840a78be5a79c7ce6bb1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } }
I20260806 03:41:21.477483 16627 raft_consensus.cc:729] T 0090c4be7d27488d87628d5297d54b72 P 29aaff6107c840a78be5a79c7ce6bb1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29aaff6107c840a78be5a79c7ce6bb1e, State: Initialized, Role: FOLLOWER
I20260806 03:41:21.478123 16627 consensus_queue.cc:260] T 0090c4be7d27488d87628d5297d54b72 P 29aaff6107c840a78be5a79c7ce6bb1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } }
I20260806 03:41:21.480530 16627 ts_tablet_manager.cc:1428] T 0090c4be7d27488d87628d5297d54b72 P 29aaff6107c840a78be5a79c7ce6bb1e: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260806 03:41:21.480650 16628 tablet_bootstrap.cc:654] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:21.487481 16628 tablet_bootstrap.cc:492] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62: No bootstrap required, opened a new log
I20260806 03:41:21.487941 16628 ts_tablet_manager.cc:1397] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.006s
I20260806 03:41:21.490697 16628 raft_consensus.cc:348] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } }
I20260806 03:41:21.491564 16628 raft_consensus.cc:729] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41affeb1e1c24f0a93ec9d5a02fe4c62, State: Initialized, Role: FOLLOWER
I20260806 03:41:21.493379 16628 consensus_queue.cc:260] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } }
I20260806 03:41:21.498186 16628 ts_tablet_manager.cc:1428] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62: Time spent starting tablet: real 0.010s user 0.004s sys 0.004s
I20260806 03:41:21.697592 16369 ts_tablet_manager-itest.cc:367] Electing peer 2...
I20260806 03:41:21.698101 16369 raft_consensus.cc:412] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 0 FOLLOWER]: Emulating election...
I20260806 03:41:21.698725 16369 raft_consensus.cc:3037] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:21.703473 16369 raft_consensus.cc:686] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 1 LEADER]: Becoming Leader. State: Replica: 41affeb1e1c24f0a93ec9d5a02fe4c62, State: Running, Role: LEADER
I20260806 03:41:21.704280 16369 consensus_queue.cc:237] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } }
I20260806 03:41:21.710198 16369 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260806 03:41:21.717491 16406 catalog_manager.cc:5495] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41affeb1e1c24f0a93ec9d5a02fe4c62 (127.15.252.66). New cstate: current_term: 1 leader_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } health_report { overall_health: HEALTHY } } }
I20260806 03:41:21.742887 16369 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260806 03:41:21.847467 16369 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260806 03:41:22.052304 16369 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260806 03:41:22.193003 16510 raft_consensus.cc:3037] T 0090c4be7d27488d87628d5297d54b72 P 29aaff6107c840a78be5a79c7ce6bb1e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:22.204350 16633 consensus_queue.cc:1035] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:41:22.356899 16369 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260806 03:41:22.357201 16369 raft_consensus.cc:412] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 0 FOLLOWER]: Emulating election...
I20260806 03:41:22.357473 16369 raft_consensus.cc:3037] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:22.363281 16369 raft_consensus.cc:686] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 1 LEADER]: Becoming Leader. State: Replica: 41affeb1e1c24f0a93ec9d5a02fe4c62, State: Running, Role: LEADER
I20260806 03:41:22.363950 16369 consensus_queue.cc:237] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } }
I20260806 03:41:22.366518 16369 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260806 03:41:22.371161 16369 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260806 03:41:22.371881 16406 catalog_manager.cc:5495] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41affeb1e1c24f0a93ec9d5a02fe4c62 (127.15.252.66). New cstate: current_term: 1 leader_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } health_report { overall_health: HEALTHY } } }
I20260806 03:41:22.476490 16369 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
W20260806 03:41:22.598807 16607 debug-util.cc:398] Leaking SignalData structure 0x7b0800133060 after lost signal to thread 16372
W20260806 03:41:22.599640 16607 debug-util.cc:398] Leaking SignalData structure 0x7b08001337c0 after lost signal to thread 16441
I20260806 03:41:22.681243 16369 cluster_itest_util.cc:256] Not converged past 0 yet: 0.0 1.1
I20260806 03:41:22.808176 16510 raft_consensus.cc:3037] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:22.816342 16644 consensus_queue.cc:1035] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 03:41:23.186239 16369 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260806 03:41:23.186532 16369 raft_consensus.cc:412] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 1 LEADER]: Emulating election...
I20260806 03:41:23.186836 16369 raft_consensus.cc:3032] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 1 LEADER]: Stepping down as leader of term 1
I20260806 03:41:23.187093 16369 raft_consensus.cc:729] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41affeb1e1c24f0a93ec9d5a02fe4c62, State: Running, Role: LEADER
I20260806 03:41:23.187599 16369 consensus_queue.cc:260] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } }
I20260806 03:41:23.188439 16369 raft_consensus.cc:3037] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:41:23.193225 16369 raft_consensus.cc:686] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 2 LEADER]: Becoming Leader. State: Replica: 41affeb1e1c24f0a93ec9d5a02fe4c62, State: Running, Role: LEADER
I20260806 03:41:23.194048 16369 consensus_queue.cc:237] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } }
I20260806 03:41:23.195747 16369 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260806 03:41:23.199765 16369 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260806 03:41:23.200436 16405 catalog_manager.cc:5495] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } health_report { overall_health: HEALTHY } } }
I20260806 03:41:23.304147 16369 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260806 03:41:23.508836 16369 cluster_itest_util.cc:256] Not converged past 0 yet: 1.1 2.2
I20260806 03:41:23.588053 16510 raft_consensus.cc:3037] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:41:23.593420 16510 raft_consensus.cc:1261] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e [term 2 FOLLOWER]: Refusing update from remote peer 41affeb1e1c24f0a93ec9d5a02fe4c62: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260806 03:41:23.595024 16652 consensus_queue.cc:1035] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260806 03:41:23.813326 16369 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260806 03:41:23.813668 16369 raft_consensus.cc:412] T 0090c4be7d27488d87628d5297d54b72 P 29aaff6107c840a78be5a79c7ce6bb1e [term 1 FOLLOWER]: Emulating election...
I20260806 03:41:23.813967 16369 raft_consensus.cc:3037] T 0090c4be7d27488d87628d5297d54b72 P 29aaff6107c840a78be5a79c7ce6bb1e [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:41:23.821134 16369 raft_consensus.cc:686] T 0090c4be7d27488d87628d5297d54b72 P 29aaff6107c840a78be5a79c7ce6bb1e [term 2 LEADER]: Becoming Leader. State: Replica: 29aaff6107c840a78be5a79c7ce6bb1e, State: Running, Role: LEADER
I20260806 03:41:23.822069 16369 consensus_queue.cc:237] T 0090c4be7d27488d87628d5297d54b72 P 29aaff6107c840a78be5a79c7ce6bb1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } }
I20260806 03:41:23.826536 16369 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260806 03:41:23.830348 16369 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260806 03:41:23.829643 16404 catalog_manager.cc:5495] T 0090c4be7d27488d87628d5297d54b72 P 29aaff6107c840a78be5a79c7ce6bb1e reported cstate change: term changed from 1 to 2, leader changed from 41affeb1e1c24f0a93ec9d5a02fe4c62 (127.15.252.66) to 29aaff6107c840a78be5a79c7ce6bb1e (127.15.252.65). New cstate: current_term: 2 leader_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } health_report { overall_health: UNKNOWN } } }
I20260806 03:41:23.934715 16369 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260806 03:41:24.101651 16510 raft_consensus.cc:1226] T 0090c4be7d27488d87628d5297d54b72 P 29aaff6107c840a78be5a79c7ce6bb1e [term 2 LEADER]: Rejecting Update request from peer 41affeb1e1c24f0a93ec9d5a02fe4c62 for earlier term 1. Current term is 2. Ops: []
I20260806 03:41:24.102960 16652 consensus_queue.cc:1046] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260806 03:41:24.103868 16652 consensus_queue.cc:1225] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [LEADER]: Peer 29aaff6107c840a78be5a79c7ce6bb1e log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260806 03:41:24.104398 16659 raft_consensus.cc:3032] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 1 LEADER]: Stepping down as leader of term 1
I20260806 03:41:24.104705 16659 raft_consensus.cc:729] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41affeb1e1c24f0a93ec9d5a02fe4c62, State: Running, Role: LEADER
I20260806 03:41:24.105238 16659 consensus_queue.cc:260] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } }
I20260806 03:41:24.106024 16659 raft_consensus.cc:3037] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:41:24.139406 16369 cluster_itest_util.cc:256] Not converged past 0 yet: 2.2 1.1
I20260806 03:41:24.436103 16586 raft_consensus.cc:1261] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 2 FOLLOWER]: Refusing update from remote peer 29aaff6107c840a78be5a79c7ce6bb1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260806 03:41:24.437599 16666 consensus_queue.cc:1035] T 0090c4be7d27488d87628d5297d54b72 P 29aaff6107c840a78be5a79c7ce6bb1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260806 03:41:24.645337 16369 ts_tablet_manager-itest.cc:367] Electing peer 1...
I20260806 03:41:24.645655 16369 raft_consensus.cc:412] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e [term 2 FOLLOWER]: Emulating election...
I20260806 03:41:24.645917 16369 raft_consensus.cc:3037] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e [term 2 FOLLOWER]: Advancing to term 3
I20260806 03:41:24.650184 16369 raft_consensus.cc:686] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e [term 3 LEADER]: Becoming Leader. State: Replica: 29aaff6107c840a78be5a79c7ce6bb1e, State: Running, Role: LEADER
I20260806 03:41:24.650910 16369 consensus_queue.cc:237] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } }
I20260806 03:41:24.653153 16369 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260806 03:41:24.658466 16369 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260806 03:41:24.658303 16404 catalog_manager.cc:5495] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e reported cstate change: term changed from 2 to 3, leader changed from 41affeb1e1c24f0a93ec9d5a02fe4c62 (127.15.252.66) to 29aaff6107c840a78be5a79c7ce6bb1e (127.15.252.65). New cstate: current_term: 3 leader_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } health_report { overall_health: UNKNOWN } } }
I20260806 03:41:24.763329 16369 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260806 03:41:24.968374 16369 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 2.2
I20260806 03:41:25.036027 16586 raft_consensus.cc:3032] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 2 LEADER]: Stepping down as leader of term 2
I20260806 03:41:25.036410 16586 raft_consensus.cc:729] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 41affeb1e1c24f0a93ec9d5a02fe4c62, State: Running, Role: LEADER
I20260806 03:41:25.036969 16586 consensus_queue.cc:260] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } }
I20260806 03:41:25.037740 16586 raft_consensus.cc:3037] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 2 FOLLOWER]: Advancing to term 3
I20260806 03:41:25.042953 16586 raft_consensus.cc:1261] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 3 FOLLOWER]: Refusing update from remote peer 29aaff6107c840a78be5a79c7ce6bb1e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260806 03:41:25.044498 16670 consensus_queue.cc:1035] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260806 03:41:25.272642 16369 ts_tablet_manager-itest.cc:367] Electing peer 3...
I20260806 03:41:25.272964 16369 raft_consensus.cc:412] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 3 FOLLOWER]: Emulating election...
I20260806 03:41:25.273257 16369 raft_consensus.cc:3037] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 3 FOLLOWER]: Advancing to term 4
I20260806 03:41:25.278546 16369 raft_consensus.cc:686] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 4 LEADER]: Becoming Leader. State: Replica: 41affeb1e1c24f0a93ec9d5a02fe4c62, State: Running, Role: LEADER
I20260806 03:41:25.279260 16369 consensus_queue.cc:237] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } }
I20260806 03:41:25.280781 16369 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260806 03:41:25.284763 16369 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260806 03:41:25.284524 16405 catalog_manager.cc:5495] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 reported cstate change: term changed from 3 to 4, leader changed from 29aaff6107c840a78be5a79c7ce6bb1e (127.15.252.65) to 41affeb1e1c24f0a93ec9d5a02fe4c62 (127.15.252.66). New cstate: current_term: 4 leader_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } health_report { overall_health: HEALTHY } } }
I20260806 03:41:25.389010 16369 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260806 03:41:25.593523 16369 cluster_itest_util.cc:256] Not converged past 0 yet: 3.3 4.4
I20260806 03:41:25.627159 16586 raft_consensus.cc:1226] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 4 LEADER]: Rejecting Update request from peer 29aaff6107c840a78be5a79c7ce6bb1e for earlier term 3. Current term is 4. Ops: []
I20260806 03:41:25.628633 16670 consensus_queue.cc:1046] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260806 03:41:25.630081 16670 consensus_queue.cc:1225] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e [LEADER]: Peer 41affeb1e1c24f0a93ec9d5a02fe4c62 log is divergent from this leader: its last log entry 4.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260806 03:41:25.630709 16684 raft_consensus.cc:3032] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e [term 3 LEADER]: Stepping down as leader of term 3
I20260806 03:41:25.631072 16684 raft_consensus.cc:729] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 29aaff6107c840a78be5a79c7ce6bb1e, State: Running, Role: LEADER
I20260806 03:41:25.631853 16684 consensus_queue.cc:260] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 } } peers { permanent_uuid: "41affeb1e1c24f0a93ec9d5a02fe4c62" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43771 } }
I20260806 03:41:25.632963 16684 raft_consensus.cc:3037] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e [term 3 FOLLOWER]: Advancing to term 4
I20260806 03:41:25.753361 16510 raft_consensus.cc:1261] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e [term 4 FOLLOWER]: Refusing update from remote peer 41affeb1e1c24f0a93ec9d5a02fe4c62: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260806 03:41:25.754880 16659 consensus_queue.cc:1035] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29aaff6107c840a78be5a79c7ce6bb1e" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 32915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260806 03:41:26.101213 16369 tablet_server.cc:178] TabletServer@127.15.252.65:0 shutting down...
I20260806 03:41:26.126114 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:26.126936 16369 tablet_replica.cc:330] T 0090c4be7d27488d87628d5297d54b72 P 29aaff6107c840a78be5a79c7ce6bb1e: stopping tablet replica
I20260806 03:41:26.127705 16369 raft_consensus.cc:2229] T 0090c4be7d27488d87628d5297d54b72 P 29aaff6107c840a78be5a79c7ce6bb1e [term 2 LEADER]: Raft consensus shutting down.
I20260806 03:41:26.128499 16369 raft_consensus.cc:2258] T 0090c4be7d27488d87628d5297d54b72 P 29aaff6107c840a78be5a79c7ce6bb1e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:26.130828 16369 tablet_replica.cc:330] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e: stopping tablet replica
I20260806 03:41:26.131377 16369 raft_consensus.cc:2229] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260806 03:41:26.131803 16369 raft_consensus.cc:2258] T 6284bbe65e3348bb9f9e585b9af87197 P 29aaff6107c840a78be5a79c7ce6bb1e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:26.287742 16369 tablet_server.cc:194] TabletServer@127.15.252.65:0 shutdown complete.
I20260806 03:41:26.299615 16369 tablet_server.cc:178] TabletServer@127.15.252.66:0 shutting down...
I20260806 03:41:26.319209 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:26.319954 16369 tablet_replica.cc:330] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62: stopping tablet replica
I20260806 03:41:26.320654 16369 raft_consensus.cc:2229] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260806 03:41:26.321148 16369 raft_consensus.cc:2258] T 0090c4be7d27488d87628d5297d54b72 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:26.322813 16369 tablet_replica.cc:330] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62: stopping tablet replica
I20260806 03:41:26.323316 16369 raft_consensus.cc:2229] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 4 LEADER]: Raft consensus shutting down.
I20260806 03:41:26.323962 16369 raft_consensus.cc:2258] T 6284bbe65e3348bb9f9e585b9af87197 P 41affeb1e1c24f0a93ec9d5a02fe4c62 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:26.366529 16369 tablet_server.cc:194] TabletServer@127.15.252.66:0 shutdown complete.
I20260806 03:41:26.375510 16369 master.cc:537] Master@127.15.252.126:46735 shutting down...
I20260806 03:41:26.765242 16369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:41:26.765849 16369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:26.766223 16369 tablet_replica.cc:330] T 00000000000000000000000000000000 P 53ca9e2083054640a7fc3f22a094277e: stopping tablet replica
I20260806 03:41:26.783747 16369 master.cc:552] Master@127.15.252.126:46735 shutdown complete.
[ OK ] LeadershipChangeReportingTest.TestConcurrentGetTableLocations (8011 ms)
[ RUN ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange
I20260806 03:41:26.816799 16369 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.252.126:38997
I20260806 03:41:26.817901 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:26.823256 16692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:26.824730 16693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:26.826779 16369 server_base.cc:1034] running on GCE node
W20260806 03:41:26.827298 16695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:26.828080 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:26.828408 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:26.828604 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987686828587 us; error 0 us; skew 500 ppm
I20260806 03:41:26.829231 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:26.836809 16369 webserver.cc:460] Webserver started at http://127.15.252.126:36511/ using document root <none> and password file <none>
I20260806 03:41:26.837371 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:26.837598 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:26.837929 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:26.839162 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1785987678583700-16369-0/minicluster-data/master-0-root/instance:
uuid: "d4891f6110ba4f6baed11a608e0fcd3f"
format_stamp: "Formatted at 2026-08-06 03:41:26 on dist-test-slave-csg5"
I20260806 03:41:26.843986 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 03:41:26.847522 16700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:26.848243 16369 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 03:41:26.848528 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1785987678583700-16369-0/minicluster-data/master-0-root
uuid: "d4891f6110ba4f6baed11a608e0fcd3f"
format_stamp: "Formatted at 2026-08-06 03:41:26 on dist-test-slave-csg5"
I20260806 03:41:26.848866 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1785987678583700-16369-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1785987678583700-16369-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1785987678583700-16369-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:26.860109 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:26.861141 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:26.898197 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.126:38997
I20260806 03:41:26.898311 16752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.126:38997 every 8 connection(s)
I20260806 03:41:26.902123 16753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:26.912220 16753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f: Bootstrap starting.
I20260806 03:41:26.916712 16753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:26.921044 16753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f: No bootstrap required, opened a new log
I20260806 03:41:26.924021 16753 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4891f6110ba4f6baed11a608e0fcd3f" member_type: VOTER }
I20260806 03:41:26.924583 16753 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:26.924983 16753 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4891f6110ba4f6baed11a608e0fcd3f, State: Initialized, Role: FOLLOWER
I20260806 03:41:26.925737 16753 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4891f6110ba4f6baed11a608e0fcd3f" member_type: VOTER }
I20260806 03:41:26.926219 16753 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:41:26.926510 16753 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:41:26.926900 16753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:26.931274 16753 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4891f6110ba4f6baed11a608e0fcd3f" member_type: VOTER }
I20260806 03:41:26.931885 16753 leader_election.cc:302] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4891f6110ba4f6baed11a608e0fcd3f; no voters:
I20260806 03:41:26.933405 16753 leader_election.cc:288] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:41:26.933917 16756 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:26.935457 16756 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f [term 1 LEADER]: Becoming Leader. State: Replica: d4891f6110ba4f6baed11a608e0fcd3f, State: Running, Role: LEADER
I20260806 03:41:26.936182 16756 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4891f6110ba4f6baed11a608e0fcd3f" member_type: VOTER }
I20260806 03:41:26.936774 16753 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:41:26.940332 16757 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4891f6110ba4f6baed11a608e0fcd3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4891f6110ba4f6baed11a608e0fcd3f" member_type: VOTER } }
I20260806 03:41:26.940966 16757 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:26.941088 16758 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4891f6110ba4f6baed11a608e0fcd3f. Latest consensus state: current_term: 1 leader_uuid: "d4891f6110ba4f6baed11a608e0fcd3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4891f6110ba4f6baed11a608e0fcd3f" member_type: VOTER } }
I20260806 03:41:26.942009 16758 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:26.944944 16763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:41:26.952559 16763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:41:26.953610 16369 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260806 03:41:26.962435 16763 catalog_manager.cc:1347] Generated new cluster ID: 88db888dbd0d49ae922e8316a55c05cc
I20260806 03:41:26.962805 16763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:41:26.987646 16763 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:41:26.989483 16763 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:41:27.021855 16763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f: Generated new TSK 0
I20260806 03:41:27.022703 16763 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:41:27.084801 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:27.090955 16774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:27.092175 16775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:27.094089 16369 server_base.cc:1034] running on GCE node
W20260806 03:41:27.093940 16777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:27.095372 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:27.095577 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:27.095702 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987687095692 us; error 0 us; skew 500 ppm
I20260806 03:41:27.096210 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:27.098884 16369 webserver.cc:460] Webserver started at http://127.15.252.65:45131/ using document root <none> and password file <none>
I20260806 03:41:27.099354 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:27.099516 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:27.099766 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:27.100907 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1785987678583700-16369-0/minicluster-data/ts-0-root/instance:
uuid: "498d41ca153c407ba701541e3edfd3c1"
format_stamp: "Formatted at 2026-08-06 03:41:27 on dist-test-slave-csg5"
I20260806 03:41:27.105676 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 03:41:27.109050 16782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:27.109972 16369 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 03:41:27.110286 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1785987678583700-16369-0/minicluster-data/ts-0-root
uuid: "498d41ca153c407ba701541e3edfd3c1"
format_stamp: "Formatted at 2026-08-06 03:41:27 on dist-test-slave-csg5"
I20260806 03:41:27.110571 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1785987678583700-16369-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1785987678583700-16369-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1785987678583700-16369-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:27.130707 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:27.131880 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:27.133360 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:27.135597 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:27.135833 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:27.136085 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:27.136250 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:27.177783 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.65:36491
I20260806 03:41:27.177882 16845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.65:36491 every 8 connection(s)
I20260806 03:41:27.182111 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:27.189340 16850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:27.191368 16851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:27.193154 16853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:27.194190 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:27.195247 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:27.195429 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:27.195538 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987687195530 us; error 0 us; skew 500 ppm
I20260806 03:41:27.196262 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:27.198767 16846 heartbeater.cc:347] Connected to a master server at 127.15.252.126:38997
I20260806 03:41:27.199132 16846 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:27.199515 16369 webserver.cc:460] Webserver started at http://127.15.252.66:38929/ using document root <none> and password file <none>
I20260806 03:41:27.199873 16846 heartbeater.cc:511] Master 127.15.252.126:38997 requested a full tablet report, sending...
I20260806 03:41:27.200026 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:27.200314 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:27.200593 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:27.201856 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1785987678583700-16369-0/minicluster-data/ts-1-root/instance:
uuid: "3f0e2aab818e4cfabd2c7d357ddba3d0"
format_stamp: "Formatted at 2026-08-06 03:41:27 on dist-test-slave-csg5"
I20260806 03:41:27.202028 16717 ts_manager.cc:194] Registered new tserver with Master: 498d41ca153c407ba701541e3edfd3c1 (127.15.252.65:36491)
I20260806 03:41:27.204201 16717 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33834
I20260806 03:41:27.207041 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 03:41:27.211000 16858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:27.211998 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 03:41:27.212358 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1785987678583700-16369-0/minicluster-data/ts-1-root
uuid: "3f0e2aab818e4cfabd2c7d357ddba3d0"
format_stamp: "Formatted at 2026-08-06 03:41:27 on dist-test-slave-csg5"
I20260806 03:41:27.212723 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1785987678583700-16369-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1785987678583700-16369-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange.1785987678583700-16369-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:27.240224 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:27.241254 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:27.242615 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:27.244916 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:27.245090 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:27.245365 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:27.245554 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:27.286020 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.66:39179
I20260806 03:41:27.286136 16921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.66:39179 every 8 connection(s)
I20260806 03:41:27.302105 16922 heartbeater.cc:347] Connected to a master server at 127.15.252.126:38997
I20260806 03:41:27.302719 16922 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:27.303449 16922 heartbeater.cc:511] Master 127.15.252.126:38997 requested a full tablet report, sending...
I20260806 03:41:27.305521 16717 ts_manager.cc:194] Registered new tserver with Master: 3f0e2aab818e4cfabd2c7d357ddba3d0 (127.15.252.66:39179)
I20260806 03:41:27.306188 16369 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.016899623s
I20260806 03:41:27.307426 16717 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33840
I20260806 03:41:27.307504 16846 heartbeater.cc:503] Master 127.15.252.126:38997 was elected leader, sending a full tablet report...
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:421: Skipped
test is skipped; set KUDU_ALLOW_SLOW_TESTS=1 to run
I20260806 03:41:27.333320 16369 tablet_server.cc:178] TabletServer@127.15.252.65:0 shutting down...
I20260806 03:41:27.370479 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:27.387734 16369 tablet_server.cc:194] TabletServer@127.15.252.65:0 shutdown complete.
I20260806 03:41:27.394886 16369 tablet_server.cc:178] TabletServer@127.15.252.66:0 shutting down...
I20260806 03:41:27.410732 16922 heartbeater.cc:503] Master 127.15.252.126:38997 was elected leader, sending a full tablet report...
I20260806 03:41:27.414584 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:27.431342 16369 tablet_server.cc:194] TabletServer@127.15.252.66:0 shutdown complete.
I20260806 03:41:27.438617 16369 master.cc:537] Master@127.15.252.126:38997 shutting down...
I20260806 03:41:27.455011 16369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:41:27.455577 16369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:27.456048 16369 tablet_replica.cc:330] T 00000000000000000000000000000000 P d4891f6110ba4f6baed11a608e0fcd3f: stopping tablet replica
I20260806 03:41:27.465548 16369 master.cc:552] Master@127.15.252.126:38997 shutdown complete.
[ SKIPPED ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange (671 ms)
[ RUN ] LeadershipChangeReportingTest.TestUpdatedConsensusState
I20260806 03:41:27.488493 16369 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.252.126:46357
I20260806 03:41:27.489528 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:27.494710 16935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:27.495756 16936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:27.497630 16938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:27.498438 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:27.499375 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:27.499538 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:27.499727 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987687499714 us; error 0 us; skew 500 ppm
I20260806 03:41:27.500272 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:27.502844 16369 webserver.cc:460] Webserver started at http://127.15.252.126:35651/ using document root <none> and password file <none>
I20260806 03:41:27.503329 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:27.503509 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:27.503795 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:27.504866 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1785987678583700-16369-0/minicluster-data/master-0-root/instance:
uuid: "7f7b19e0ec484106898d629cc3078a3a"
format_stamp: "Formatted at 2026-08-06 03:41:27 on dist-test-slave-csg5"
I20260806 03:41:27.509341 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 03:41:27.513193 16943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:27.514016 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 03:41:27.514338 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1785987678583700-16369-0/minicluster-data/master-0-root
uuid: "7f7b19e0ec484106898d629cc3078a3a"
format_stamp: "Formatted at 2026-08-06 03:41:27 on dist-test-slave-csg5"
I20260806 03:41:27.514659 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1785987678583700-16369-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1785987678583700-16369-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1785987678583700-16369-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:27.536484 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:27.537530 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:27.575341 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.126:46357
I20260806 03:41:27.575476 16994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.126:46357 every 8 connection(s)
I20260806 03:41:27.579087 16995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:27.588652 16995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a: Bootstrap starting.
I20260806 03:41:27.592571 16995 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:27.596849 16995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a: No bootstrap required, opened a new log
I20260806 03:41:27.598729 16995 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7b19e0ec484106898d629cc3078a3a" member_type: VOTER }
I20260806 03:41:27.599120 16995 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:27.599404 16995 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f7b19e0ec484106898d629cc3078a3a, State: Initialized, Role: FOLLOWER
I20260806 03:41:27.599942 16995 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7b19e0ec484106898d629cc3078a3a" member_type: VOTER }
I20260806 03:41:27.600411 16995 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:41:27.600659 16995 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:41:27.600922 16995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:27.605162 16995 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7b19e0ec484106898d629cc3078a3a" member_type: VOTER }
I20260806 03:41:27.605718 16995 leader_election.cc:302] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f7b19e0ec484106898d629cc3078a3a; no voters:
I20260806 03:41:27.607007 16995 leader_election.cc:288] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:41:27.607374 16998 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:27.608975 16998 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a [term 1 LEADER]: Becoming Leader. State: Replica: 7f7b19e0ec484106898d629cc3078a3a, State: Running, Role: LEADER
I20260806 03:41:27.609864 16995 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:41:27.609753 16998 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7b19e0ec484106898d629cc3078a3a" member_type: VOTER }
I20260806 03:41:27.612794 17000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f7b19e0ec484106898d629cc3078a3a. Latest consensus state: current_term: 1 leader_uuid: "7f7b19e0ec484106898d629cc3078a3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7b19e0ec484106898d629cc3078a3a" member_type: VOTER } }
I20260806 03:41:27.612722 16999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f7b19e0ec484106898d629cc3078a3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f7b19e0ec484106898d629cc3078a3a" member_type: VOTER } }
I20260806 03:41:27.613314 17000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:27.613359 16999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:27.618276 17003 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:41:27.623011 17003 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:41:27.625528 16369 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260806 03:41:27.631584 17003 catalog_manager.cc:1347] Generated new cluster ID: 151fd00a4af0480685357144766e3ddf
I20260806 03:41:27.631882 17003 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:41:27.650406 17003 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:41:27.652096 17003 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:41:27.667030 17003 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a: Generated new TSK 0
I20260806 03:41:27.667626 17003 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:41:27.692716 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:27.699241 17016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:27.700814 17017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:27.702100 16369 server_base.cc:1034] running on GCE node
W20260806 03:41:27.702068 17019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:27.703229 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:27.703430 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:27.703544 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987687703536 us; error 0 us; skew 500 ppm
I20260806 03:41:27.703994 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:27.706588 16369 webserver.cc:460] Webserver started at http://127.15.252.65:35631/ using document root <none> and password file <none>
I20260806 03:41:27.707149 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:27.707314 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:27.707526 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:27.708604 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1785987678583700-16369-0/minicluster-data/ts-0-root/instance:
uuid: "2aa8fa6179524a80ac82cf5433138568"
format_stamp: "Formatted at 2026-08-06 03:41:27 on dist-test-slave-csg5"
I20260806 03:41:27.713542 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 03:41:27.717188 17024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:27.718138 16369 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260806 03:41:27.718456 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1785987678583700-16369-0/minicluster-data/ts-0-root
uuid: "2aa8fa6179524a80ac82cf5433138568"
format_stamp: "Formatted at 2026-08-06 03:41:27 on dist-test-slave-csg5"
I20260806 03:41:27.718868 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1785987678583700-16369-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1785987678583700-16369-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1785987678583700-16369-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:27.734622 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:27.735920 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:27.737604 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:27.740139 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:27.740397 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:27.740685 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:27.740869 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:27.784090 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.65:35003
I20260806 03:41:27.784175 17087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.65:35003 every 8 connection(s)
I20260806 03:41:27.789194 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:27.797416 17092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:27.798168 17093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:27.800844 17095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:27.801375 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:27.802153 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:27.802390 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:27.802588 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987687802572 us; error 0 us; skew 500 ppm
I20260806 03:41:27.803316 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:27.806144 17088 heartbeater.cc:347] Connected to a master server at 127.15.252.126:46357
I20260806 03:41:27.806563 17088 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:27.806715 16369 webserver.cc:460] Webserver started at http://127.15.252.66:33419/ using document root <none> and password file <none>
I20260806 03:41:27.807292 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:27.807319 17088 heartbeater.cc:511] Master 127.15.252.126:46357 requested a full tablet report, sending...
I20260806 03:41:27.807560 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:27.807890 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:27.809214 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1785987678583700-16369-0/minicluster-data/ts-1-root/instance:
uuid: "cae42678de5f4a2ca32fb676fc8f84d3"
format_stamp: "Formatted at 2026-08-06 03:41:27 on dist-test-slave-csg5"
I20260806 03:41:27.809690 16960 ts_manager.cc:194] Registered new tserver with Master: 2aa8fa6179524a80ac82cf5433138568 (127.15.252.65:35003)
I20260806 03:41:27.811813 16960 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41628
I20260806 03:41:27.814378 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 03:41:27.818056 17100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:27.819281 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 03:41:27.819607 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1785987678583700-16369-0/minicluster-data/ts-1-root
uuid: "cae42678de5f4a2ca32fb676fc8f84d3"
format_stamp: "Formatted at 2026-08-06 03:41:27 on dist-test-slave-csg5"
I20260806 03:41:27.819939 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1785987678583700-16369-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1785987678583700-16369-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.LeadershipChangeReportingTest.TestUpdatedConsensusState.1785987678583700-16369-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:27.840123 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:27.841121 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:27.842564 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:27.844899 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:27.845083 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:27.845398 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:27.845556 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:27.885713 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.66:33207
I20260806 03:41:27.885812 17163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.66:33207 every 8 connection(s)
I20260806 03:41:27.903128 17164 heartbeater.cc:347] Connected to a master server at 127.15.252.126:46357
I20260806 03:41:27.903573 17164 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:27.904278 17164 heartbeater.cc:511] Master 127.15.252.126:46357 requested a full tablet report, sending...
I20260806 03:41:27.906286 16960 ts_manager.cc:194] Registered new tserver with Master: cae42678de5f4a2ca32fb676fc8f84d3 (127.15.252.66:33207)
I20260806 03:41:27.907106 16369 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.018334569s
I20260806 03:41:27.908330 16960 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41634
I20260806 03:41:27.914518 17088 heartbeater.cc:503] Master 127.15.252.126:46357 was elected leader, sending a full tablet report...
I20260806 03:41:27.934422 16960 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41638:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 2
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:41:27.936077 16960 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 3 tablet servers would be needed, 2 are available. Consider bringing up more tablet servers.
I20260806 03:41:27.987190 17052 tablet_service.cc:1467] Processing CreateTablet for tablet 7042701a9fc3459eb1e3e2f0b209f239 (DEFAULT_TABLE table=test-table [id=586cf7b4a00d4bac936e35bea7d55879]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:41:27.988538 17052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7042701a9fc3459eb1e3e2f0b209f239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:27.988586 17128 tablet_service.cc:1467] Processing CreateTablet for tablet 7042701a9fc3459eb1e3e2f0b209f239 (DEFAULT_TABLE table=test-table [id=586cf7b4a00d4bac936e35bea7d55879]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:41:27.989661 17128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7042701a9fc3459eb1e3e2f0b209f239. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:28.002023 17177 tablet_bootstrap.cc:492] T 7042701a9fc3459eb1e3e2f0b209f239 P cae42678de5f4a2ca32fb676fc8f84d3: Bootstrap starting.
I20260806 03:41:28.003093 17178 tablet_bootstrap.cc:492] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568: Bootstrap starting.
I20260806 03:41:28.007198 17177 tablet_bootstrap.cc:654] T 7042701a9fc3459eb1e3e2f0b209f239 P cae42678de5f4a2ca32fb676fc8f84d3: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:28.007529 17178 tablet_bootstrap.cc:654] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:28.011919 17164 heartbeater.cc:503] Master 127.15.252.126:46357 was elected leader, sending a full tablet report...
I20260806 03:41:28.015471 17177 tablet_bootstrap.cc:492] T 7042701a9fc3459eb1e3e2f0b209f239 P cae42678de5f4a2ca32fb676fc8f84d3: No bootstrap required, opened a new log
I20260806 03:41:28.015944 17177 ts_tablet_manager.cc:1397] T 7042701a9fc3459eb1e3e2f0b209f239 P cae42678de5f4a2ca32fb676fc8f84d3: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.008s
I20260806 03:41:28.016052 17178 tablet_bootstrap.cc:492] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568: No bootstrap required, opened a new log
I20260806 03:41:28.016518 17178 ts_tablet_manager.cc:1397] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.003s
I20260806 03:41:28.018365 17177 raft_consensus.cc:348] T 7042701a9fc3459eb1e3e2f0b209f239 P cae42678de5f4a2ca32fb676fc8f84d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aa8fa6179524a80ac82cf5433138568" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35003 } } peers { permanent_uuid: "cae42678de5f4a2ca32fb676fc8f84d3" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 33207 } }
I20260806 03:41:28.019285 17177 raft_consensus.cc:729] T 7042701a9fc3459eb1e3e2f0b209f239 P cae42678de5f4a2ca32fb676fc8f84d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cae42678de5f4a2ca32fb676fc8f84d3, State: Initialized, Role: FOLLOWER
I20260806 03:41:28.019408 17178 raft_consensus.cc:348] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aa8fa6179524a80ac82cf5433138568" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35003 } } peers { permanent_uuid: "cae42678de5f4a2ca32fb676fc8f84d3" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 33207 } }
I20260806 03:41:28.020092 17178 raft_consensus.cc:729] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2aa8fa6179524a80ac82cf5433138568, State: Initialized, Role: FOLLOWER
I20260806 03:41:28.020764 17178 consensus_queue.cc:260] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aa8fa6179524a80ac82cf5433138568" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35003 } } peers { permanent_uuid: "cae42678de5f4a2ca32fb676fc8f84d3" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 33207 } }
I20260806 03:41:28.020118 17177 consensus_queue.cc:260] T 7042701a9fc3459eb1e3e2f0b209f239 P cae42678de5f4a2ca32fb676fc8f84d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aa8fa6179524a80ac82cf5433138568" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35003 } } peers { permanent_uuid: "cae42678de5f4a2ca32fb676fc8f84d3" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 33207 } }
I20260806 03:41:28.024765 17178 ts_tablet_manager.cc:1428] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568: Time spent starting tablet: real 0.008s user 0.005s sys 0.003s
I20260806 03:41:28.026829 17177 ts_tablet_manager.cc:1428] T 7042701a9fc3459eb1e3e2f0b209f239 P cae42678de5f4a2ca32fb676fc8f84d3: Time spent starting tablet: real 0.010s user 0.004s sys 0.007s
I20260806 03:41:28.062299 16369 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260806 03:41:28.062541 16369 raft_consensus.cc:412] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [term 0 FOLLOWER]: Emulating election...
I20260806 03:41:28.062868 16369 raft_consensus.cc:3037] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:28.067329 16369 raft_consensus.cc:686] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [term 1 LEADER]: Becoming Leader. State: Replica: 2aa8fa6179524a80ac82cf5433138568, State: Running, Role: LEADER
W20260806 03:41:28.067891 17088 heartbeater.cc:644] Failed to heartbeat to 127.15.252.126:46357 (0 consecutive failures): IO error: failing all heartbeats for tests
I20260806 03:41:28.068045 16369 consensus_queue.cc:237] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aa8fa6179524a80ac82cf5433138568" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35003 } } peers { permanent_uuid: "cae42678de5f4a2ca32fb676fc8f84d3" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 33207 } }
I20260806 03:41:28.071295 16369 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260806 03:41:28.097376 16369 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0
I20260806 03:41:28.202288 16369 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0
I20260806 03:41:28.406946 16369 cluster_itest_util.cc:256] Not converged past 1 yet: 1.1 0.0
I20260806 03:41:28.461565 17138 raft_consensus.cc:3037] T 7042701a9fc3459eb1e3e2f0b209f239 P cae42678de5f4a2ca32fb676fc8f84d3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:28.467731 17181 consensus_queue.cc:1035] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cae42678de5f4a2ca32fb676fc8f84d3" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 33207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:41:28.713267 16369 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260806 03:41:28.713512 16369 raft_consensus.cc:412] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [term 1 LEADER]: Emulating election...
I20260806 03:41:28.713721 16369 raft_consensus.cc:3032] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [term 1 LEADER]: Stepping down as leader of term 1
I20260806 03:41:28.713886 16369 raft_consensus.cc:729] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 2aa8fa6179524a80ac82cf5433138568, State: Running, Role: LEADER
I20260806 03:41:28.714308 16369 consensus_queue.cc:260] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aa8fa6179524a80ac82cf5433138568" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35003 } } peers { permanent_uuid: "cae42678de5f4a2ca32fb676fc8f84d3" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 33207 } }
I20260806 03:41:28.715128 16369 raft_consensus.cc:3037] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:41:28.720222 16369 raft_consensus.cc:686] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [term 2 LEADER]: Becoming Leader. State: Replica: 2aa8fa6179524a80ac82cf5433138568, State: Running, Role: LEADER
I20260806 03:41:28.721081 16369 consensus_queue.cc:237] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aa8fa6179524a80ac82cf5433138568" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35003 } } peers { permanent_uuid: "cae42678de5f4a2ca32fb676fc8f84d3" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 33207 } }
I20260806 03:41:28.722851 16369 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260806 03:41:28.726991 16369 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260806 03:41:28.845098 16369 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260806 03:41:29.050092 16369 cluster_itest_util.cc:256] Not converged past 2 yet: 2.2 1.1
I20260806 03:41:29.314555 17138 raft_consensus.cc:3037] T 7042701a9fc3459eb1e3e2f0b209f239 P cae42678de5f4a2ca32fb676fc8f84d3 [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:41:29.321429 17138 raft_consensus.cc:1261] T 7042701a9fc3459eb1e3e2f0b209f239 P cae42678de5f4a2ca32fb676fc8f84d3 [term 2 FOLLOWER]: Refusing update from remote peer 2aa8fa6179524a80ac82cf5433138568: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260806 03:41:29.322949 17191 consensus_queue.cc:1035] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cae42678de5f4a2ca32fb676fc8f84d3" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 33207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260806 03:41:29.356334 16369 ts_tablet_manager-itest.cc:367] Electing peer 0...
I20260806 03:41:29.356609 16369 raft_consensus.cc:412] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [term 2 LEADER]: Emulating election...
I20260806 03:41:29.356881 16369 raft_consensus.cc:3032] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [term 2 LEADER]: Stepping down as leader of term 2
I20260806 03:41:29.357122 16369 raft_consensus.cc:729] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 2aa8fa6179524a80ac82cf5433138568, State: Running, Role: LEADER
I20260806 03:41:29.357661 16369 consensus_queue.cc:260] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aa8fa6179524a80ac82cf5433138568" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35003 } } peers { permanent_uuid: "cae42678de5f4a2ca32fb676fc8f84d3" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 33207 } }
I20260806 03:41:29.358476 16369 raft_consensus.cc:3037] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [term 2 FOLLOWER]: Advancing to term 3
I20260806 03:41:29.363910 16369 raft_consensus.cc:686] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [term 3 LEADER]: Becoming Leader. State: Replica: 2aa8fa6179524a80ac82cf5433138568, State: Running, Role: LEADER
I20260806 03:41:29.364547 16369 consensus_queue.cc:237] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aa8fa6179524a80ac82cf5433138568" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35003 } } peers { permanent_uuid: "cae42678de5f4a2ca32fb676fc8f84d3" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 33207 } }
I20260806 03:41:29.366086 16369 ts_tablet_manager-itest.cc:370] Waiting for servers to agree...
I20260806 03:41:29.370787 16369 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260806 03:41:29.475888 16369 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260806 03:41:29.680236 16369 cluster_itest_util.cc:256] Not converged past 3 yet: 3.3 2.2
I20260806 03:41:29.803309 17138 raft_consensus.cc:3037] T 7042701a9fc3459eb1e3e2f0b209f239 P cae42678de5f4a2ca32fb676fc8f84d3 [term 2 FOLLOWER]: Advancing to term 3
I20260806 03:41:29.808254 17138 raft_consensus.cc:1261] T 7042701a9fc3459eb1e3e2f0b209f239 P cae42678de5f4a2ca32fb676fc8f84d3 [term 3 FOLLOWER]: Refusing update from remote peer 2aa8fa6179524a80ac82cf5433138568: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260806 03:41:29.809639 17193 consensus_queue.cc:1035] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cae42678de5f4a2ca32fb676fc8f84d3" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 33207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260806 03:41:29.989717 16369 tablet_server.cc:178] TabletServer@127.15.252.65:0 shutting down...
I20260806 03:41:30.013665 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:30.014604 16369 tablet_replica.cc:330] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568: stopping tablet replica
I20260806 03:41:30.015242 16369 raft_consensus.cc:2229] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [term 3 LEADER]: Raft consensus shutting down.
I20260806 03:41:30.015960 16369 raft_consensus.cc:2258] T 7042701a9fc3459eb1e3e2f0b209f239 P 2aa8fa6179524a80ac82cf5433138568 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:30.039149 16369 tablet_server.cc:194] TabletServer@127.15.252.65:0 shutdown complete.
I20260806 03:41:30.050168 16369 tablet_server.cc:178] TabletServer@127.15.252.66:0 shutting down...
I20260806 03:41:30.077860 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:30.078871 16369 tablet_replica.cc:330] T 7042701a9fc3459eb1e3e2f0b209f239 P cae42678de5f4a2ca32fb676fc8f84d3: stopping tablet replica
I20260806 03:41:30.079406 16369 raft_consensus.cc:2229] T 7042701a9fc3459eb1e3e2f0b209f239 P cae42678de5f4a2ca32fb676fc8f84d3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260806 03:41:30.079905 16369 raft_consensus.cc:2258] T 7042701a9fc3459eb1e3e2f0b209f239 P cae42678de5f4a2ca32fb676fc8f84d3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:30.100381 16369 tablet_server.cc:194] TabletServer@127.15.252.66:0 shutdown complete.
I20260806 03:41:30.109232 16369 master.cc:537] Master@127.15.252.126:46357 shutting down...
I20260806 03:41:30.131476 16369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:41:30.132148 16369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:30.132459 16369 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7f7b19e0ec484106898d629cc3078a3a: stopping tablet replica
I20260806 03:41:30.150419 16369 master.cc:552] Master@127.15.252.126:46357 shutdown complete.
[ OK ] LeadershipChangeReportingTest.TestUpdatedConsensusState (2695 ms)
[----------] 3 tests from LeadershipChangeReportingTest (11378 ms total)
[----------] 4 tests from TsTabletManagerITest
[ RUN ] TsTabletManagerITest.ReportOnReplicaHealthStatus
I20260806 03:41:30.184115 16369 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.252.126:36697
I20260806 03:41:30.185273 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:30.192153 17202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:30.192299 17201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:30.193389 17204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:30.194787 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:30.195691 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:30.195920 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:30.196125 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987690196112 us; error 0 us; skew 500 ppm
I20260806 03:41:30.196740 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:30.199755 16369 webserver.cc:460] Webserver started at http://127.15.252.126:38985/ using document root <none> and password file <none>
I20260806 03:41:30.200299 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:30.200459 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:30.200806 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:30.202041 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1785987678583700-16369-0/minicluster-data/master-0-root/instance:
uuid: "006341462e54415c9779495e19addadd"
format_stamp: "Formatted at 2026-08-06 03:41:30 on dist-test-slave-csg5"
I20260806 03:41:30.207769 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260806 03:41:30.211740 17209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:30.212729 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 03:41:30.213059 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1785987678583700-16369-0/minicluster-data/master-0-root
uuid: "006341462e54415c9779495e19addadd"
format_stamp: "Formatted at 2026-08-06 03:41:30 on dist-test-slave-csg5"
I20260806 03:41:30.213433 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1785987678583700-16369-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1785987678583700-16369-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1785987678583700-16369-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:30.234862 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:30.236266 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:30.282991 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.126:36697
I20260806 03:41:30.283087 17261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.126:36697 every 8 connection(s)
I20260806 03:41:30.287508 17262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:30.297703 17262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd: Bootstrap starting.
I20260806 03:41:30.302165 17262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:30.307732 17262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd: No bootstrap required, opened a new log
I20260806 03:41:30.309862 17262 raft_consensus.cc:348] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "006341462e54415c9779495e19addadd" member_type: VOTER }
I20260806 03:41:30.310277 17262 raft_consensus.cc:374] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:30.310472 17262 raft_consensus.cc:729] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 006341462e54415c9779495e19addadd, State: Initialized, Role: FOLLOWER
I20260806 03:41:30.311169 17262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "006341462e54415c9779495e19addadd" member_type: VOTER }
I20260806 03:41:30.311625 17262 raft_consensus.cc:388] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:41:30.311838 17262 raft_consensus.cc:482] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:41:30.312067 17262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:30.317132 17262 raft_consensus.cc:504] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "006341462e54415c9779495e19addadd" member_type: VOTER }
I20260806 03:41:30.317683 17262 leader_election.cc:302] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 006341462e54415c9779495e19addadd; no voters:
I20260806 03:41:30.318997 17262 leader_election.cc:288] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:41:30.319527 17265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:30.321029 17265 raft_consensus.cc:686] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd [term 1 LEADER]: Becoming Leader. State: Replica: 006341462e54415c9779495e19addadd, State: Running, Role: LEADER
I20260806 03:41:30.321633 17265 consensus_queue.cc:237] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "006341462e54415c9779495e19addadd" member_type: VOTER }
I20260806 03:41:30.322438 17262 sys_catalog.cc:564] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:41:30.324616 17267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 006341462e54415c9779495e19addadd. Latest consensus state: current_term: 1 leader_uuid: "006341462e54415c9779495e19addadd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "006341462e54415c9779495e19addadd" member_type: VOTER } }
I20260806 03:41:30.324687 17266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "006341462e54415c9779495e19addadd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "006341462e54415c9779495e19addadd" member_type: VOTER } }
I20260806 03:41:30.325312 17267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:30.325493 17266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:30.328650 17270 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:41:30.334216 17270 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:41:30.340346 16369 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260806 03:41:30.344242 17270 catalog_manager.cc:1347] Generated new cluster ID: 15475cf71a3b45febecc995c84d0d365
I20260806 03:41:30.344554 17270 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:41:30.389065 17270 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:41:30.390870 17270 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:41:30.404459 17270 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd: Generated new TSK 0
I20260806 03:41:30.405210 17270 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:41:30.472530 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:30.479655 17283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:30.481544 17284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:30.483075 17286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:30.483744 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:30.484625 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:30.484815 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:30.484931 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987690484917 us; error 0 us; skew 500 ppm
I20260806 03:41:30.485508 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:30.488134 16369 webserver.cc:460] Webserver started at http://127.15.252.65:39733/ using document root <none> and password file <none>
I20260806 03:41:30.488627 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:30.488811 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:30.489097 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:30.490320 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1785987678583700-16369-0/minicluster-data/ts-0-root/instance:
uuid: "20fed7f68964400a84869acd9962a256"
format_stamp: "Formatted at 2026-08-06 03:41:30 on dist-test-slave-csg5"
I20260806 03:41:30.494910 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 03:41:30.498510 17291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:30.499558 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260806 03:41:30.499892 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1785987678583700-16369-0/minicluster-data/ts-0-root
uuid: "20fed7f68964400a84869acd9962a256"
format_stamp: "Formatted at 2026-08-06 03:41:30 on dist-test-slave-csg5"
I20260806 03:41:30.500222 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1785987678583700-16369-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1785987678583700-16369-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1785987678583700-16369-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:30.531601 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:30.532953 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:30.534578 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:30.536878 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:30.537102 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:30.537359 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:30.537544 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:30.583909 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.65:40851
I20260806 03:41:30.584035 17356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.65:40851 every 8 connection(s)
I20260806 03:41:30.589262 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:30.596678 17361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:30.597391 17362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:30.599403 17364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:30.600610 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:30.601791 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:30.602097 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:30.602381 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987690602337 us; error 0 us; skew 500 ppm
I20260806 03:41:30.603032 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:30.606436 17357 heartbeater.cc:347] Connected to a master server at 127.15.252.126:36697
I20260806 03:41:30.606513 16369 webserver.cc:460] Webserver started at http://127.15.252.66:34541/ using document root <none> and password file <none>
I20260806 03:41:30.607009 17357 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:30.607273 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:30.607522 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:30.607853 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:30.607887 17357 heartbeater.cc:511] Master 127.15.252.126:36697 requested a full tablet report, sending...
I20260806 03:41:30.609462 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1785987678583700-16369-0/minicluster-data/ts-1-root/instance:
uuid: "f41dc769141846939903d45745819f86"
format_stamp: "Formatted at 2026-08-06 03:41:30 on dist-test-slave-csg5"
I20260806 03:41:30.610452 17226 ts_manager.cc:194] Registered new tserver with Master: 20fed7f68964400a84869acd9962a256 (127.15.252.65:40851)
I20260806 03:41:30.612619 17226 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53294
I20260806 03:41:30.615265 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260806 03:41:30.619074 17369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:30.619839 16369 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 03:41:30.620146 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1785987678583700-16369-0/minicluster-data/ts-1-root
uuid: "f41dc769141846939903d45745819f86"
format_stamp: "Formatted at 2026-08-06 03:41:30 on dist-test-slave-csg5"
I20260806 03:41:30.620464 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1785987678583700-16369-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1785987678583700-16369-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1785987678583700-16369-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:30.629510 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:30.630803 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:30.632290 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:30.634737 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:30.634963 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:30.635215 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:30.635385 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:30.685802 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.66:37197
I20260806 03:41:30.685945 17433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.66:37197 every 8 connection(s)
I20260806 03:41:30.690838 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:30.698807 17438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:30.699653 17439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:30.702039 17441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:30.703125 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:30.703948 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:30.704172 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:30.704355 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987690704339 us; error 0 us; skew 500 ppm
I20260806 03:41:30.704886 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:30.708115 16369 webserver.cc:460] Webserver started at http://127.15.252.67:37427/ using document root <none> and password file <none>
I20260806 03:41:30.708622 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:30.708837 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:30.708858 17434 heartbeater.cc:347] Connected to a master server at 127.15.252.126:36697
I20260806 03:41:30.709192 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:30.709312 17434 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:30.710115 17434 heartbeater.cc:511] Master 127.15.252.126:36697 requested a full tablet report, sending...
I20260806 03:41:30.710525 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1785987678583700-16369-0/minicluster-data/ts-2-root/instance:
uuid: "2829b5cb964d43d3b75985dfa7b19563"
format_stamp: "Formatted at 2026-08-06 03:41:30 on dist-test-slave-csg5"
I20260806 03:41:30.712345 17226 ts_manager.cc:194] Registered new tserver with Master: f41dc769141846939903d45745819f86 (127.15.252.66:37197)
I20260806 03:41:30.714617 17226 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53304
I20260806 03:41:30.715587 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 03:41:30.719697 17446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:30.720659 16369 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260806 03:41:30.720920 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1785987678583700-16369-0/minicluster-data/ts-2-root
uuid: "2829b5cb964d43d3b75985dfa7b19563"
format_stamp: "Formatted at 2026-08-06 03:41:30 on dist-test-slave-csg5"
I20260806 03:41:30.721196 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1785987678583700-16369-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1785987678583700-16369-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.ReportOnReplicaHealthStatus.1785987678583700-16369-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:30.743866 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:30.745051 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:30.746621 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:30.749004 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:30.749243 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:30.749516 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:30.749692 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:30.825104 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.67:38455
I20260806 03:41:30.825191 17509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.67:38455 every 8 connection(s)
I20260806 03:41:30.842955 17510 heartbeater.cc:347] Connected to a master server at 127.15.252.126:36697
I20260806 03:41:30.843370 17510 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:30.844149 17510 heartbeater.cc:511] Master 127.15.252.126:36697 requested a full tablet report, sending...
I20260806 03:41:30.846288 17226 ts_manager.cc:194] Registered new tserver with Master: 2829b5cb964d43d3b75985dfa7b19563 (127.15.252.67:38455)
I20260806 03:41:30.846451 16369 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018062591s
I20260806 03:41:30.847978 17226 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53310
I20260806 03:41:30.877285 17226 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53314:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
W20260806 03:41:30.879046 17226 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:41:30.943998 17320 tablet_service.cc:1467] Processing CreateTablet for tablet dca885b78222414284c7dd450e6c01e1 (DEFAULT_TABLE table=test-table [id=6aa2e7e80e104f708ead9a76f55617ea]), partition=
I20260806 03:41:30.945087 17320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dca885b78222414284c7dd450e6c01e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:30.955332 17475 tablet_service.cc:1467] Processing CreateTablet for tablet dca885b78222414284c7dd450e6c01e1 (DEFAULT_TABLE table=test-table [id=6aa2e7e80e104f708ead9a76f55617ea]), partition=
I20260806 03:41:30.956528 17475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dca885b78222414284c7dd450e6c01e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:30.962297 17398 tablet_service.cc:1467] Processing CreateTablet for tablet dca885b78222414284c7dd450e6c01e1 (DEFAULT_TABLE table=test-table [id=6aa2e7e80e104f708ead9a76f55617ea]), partition=
I20260806 03:41:30.963528 17398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dca885b78222414284c7dd450e6c01e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:30.965226 17526 tablet_bootstrap.cc:492] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256: Bootstrap starting.
I20260806 03:41:30.973086 17526 tablet_bootstrap.cc:654] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:30.981887 17527 tablet_bootstrap.cc:492] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563: Bootstrap starting.
I20260806 03:41:30.987139 17530 tablet_bootstrap.cc:492] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86: Bootstrap starting.
I20260806 03:41:30.989143 17526 tablet_bootstrap.cc:492] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256: No bootstrap required, opened a new log
I20260806 03:41:30.989225 17527 tablet_bootstrap.cc:654] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:30.989826 17526 ts_tablet_manager.cc:1397] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256: Time spent bootstrapping tablet: real 0.025s user 0.011s sys 0.004s
I20260806 03:41:30.995016 17530 tablet_bootstrap.cc:654] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:30.993721 17526 raft_consensus.cc:348] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20fed7f68964400a84869acd9962a256" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 40851 } } peers { permanent_uuid: "2829b5cb964d43d3b75985dfa7b19563" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 38455 } } peers { permanent_uuid: "f41dc769141846939903d45745819f86" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37197 } }
I20260806 03:41:30.997088 17526 raft_consensus.cc:374] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:30.997531 17526 raft_consensus.cc:729] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20fed7f68964400a84869acd9962a256, State: Initialized, Role: FOLLOWER
I20260806 03:41:30.998296 17526 consensus_queue.cc:260] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20fed7f68964400a84869acd9962a256" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 40851 } } peers { permanent_uuid: "2829b5cb964d43d3b75985dfa7b19563" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 38455 } } peers { permanent_uuid: "f41dc769141846939903d45745819f86" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37197 } }
I20260806 03:41:31.001763 17527 tablet_bootstrap.cc:492] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563: No bootstrap required, opened a new log
I20260806 03:41:31.002351 17527 ts_tablet_manager.cc:1397] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.008s
I20260806 03:41:31.004570 17357 heartbeater.cc:503] Master 127.15.252.126:36697 was elected leader, sending a full tablet report...
I20260806 03:41:31.005282 17530 tablet_bootstrap.cc:492] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86: No bootstrap required, opened a new log
I20260806 03:41:31.005420 17526 ts_tablet_manager.cc:1428] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256: Time spent starting tablet: real 0.015s user 0.007s sys 0.004s
I20260806 03:41:31.005766 17530 ts_tablet_manager.cc:1397] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.011s
I20260806 03:41:31.005534 17527 raft_consensus.cc:348] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20fed7f68964400a84869acd9962a256" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 40851 } } peers { permanent_uuid: "2829b5cb964d43d3b75985dfa7b19563" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 38455 } } peers { permanent_uuid: "f41dc769141846939903d45745819f86" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37197 } }
I20260806 03:41:31.006469 17527 raft_consensus.cc:374] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:31.006868 17527 raft_consensus.cc:729] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2829b5cb964d43d3b75985dfa7b19563, State: Initialized, Role: FOLLOWER
I20260806 03:41:31.007812 17527 consensus_queue.cc:260] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20fed7f68964400a84869acd9962a256" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 40851 } } peers { permanent_uuid: "2829b5cb964d43d3b75985dfa7b19563" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 38455 } } peers { permanent_uuid: "f41dc769141846939903d45745819f86" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37197 } }
I20260806 03:41:31.008602 17530 raft_consensus.cc:348] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20fed7f68964400a84869acd9962a256" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 40851 } } peers { permanent_uuid: "2829b5cb964d43d3b75985dfa7b19563" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 38455 } } peers { permanent_uuid: "f41dc769141846939903d45745819f86" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37197 } }
I20260806 03:41:31.009465 17530 raft_consensus.cc:374] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:31.009770 17530 raft_consensus.cc:729] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f41dc769141846939903d45745819f86, State: Initialized, Role: FOLLOWER
I20260806 03:41:31.012136 17510 heartbeater.cc:503] Master 127.15.252.126:36697 was elected leader, sending a full tablet report...
I20260806 03:41:31.012642 17527 ts_tablet_manager.cc:1428] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563: Time spent starting tablet: real 0.010s user 0.005s sys 0.004s
I20260806 03:41:31.010586 17530 consensus_queue.cc:260] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20fed7f68964400a84869acd9962a256" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 40851 } } peers { permanent_uuid: "2829b5cb964d43d3b75985dfa7b19563" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 38455 } } peers { permanent_uuid: "f41dc769141846939903d45745819f86" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37197 } }
I20260806 03:41:31.019528 17434 heartbeater.cc:503] Master 127.15.252.126:36697 was elected leader, sending a full tablet report...
I20260806 03:41:31.020215 17530 ts_tablet_manager.cc:1428] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86: Time spent starting tablet: real 0.014s user 0.009s sys 0.002s
I20260806 03:41:31.309341 17533 raft_consensus.cc:482] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:41:31.309906 17533 raft_consensus.cc:504] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20fed7f68964400a84869acd9962a256" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 40851 } } peers { permanent_uuid: "2829b5cb964d43d3b75985dfa7b19563" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 38455 } } peers { permanent_uuid: "f41dc769141846939903d45745819f86" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37197 } }
I20260806 03:41:31.312325 17533 leader_election.cc:288] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2829b5cb964d43d3b75985dfa7b19563 (127.15.252.67:38455), f41dc769141846939903d45745819f86 (127.15.252.66:37197)
I20260806 03:41:31.333514 17485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca885b78222414284c7dd450e6c01e1" candidate_uuid: "20fed7f68964400a84869acd9962a256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2829b5cb964d43d3b75985dfa7b19563" is_pre_election: true
I20260806 03:41:31.333580 17408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca885b78222414284c7dd450e6c01e1" candidate_uuid: "20fed7f68964400a84869acd9962a256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41dc769141846939903d45745819f86" is_pre_election: true
I20260806 03:41:31.334832 17485 raft_consensus.cc:2454] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20fed7f68964400a84869acd9962a256 in term 0.
I20260806 03:41:31.337242 17294 leader_election.cc:302] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20fed7f68964400a84869acd9962a256, 2829b5cb964d43d3b75985dfa7b19563; no voters:
I20260806 03:41:31.337327 17408 raft_consensus.cc:2454] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20fed7f68964400a84869acd9962a256 in term 0.
I20260806 03:41:31.338096 17533 raft_consensus.cc:2781] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:41:31.338402 17533 raft_consensus.cc:482] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:41:31.338742 17533 raft_consensus.cc:3037] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:31.343611 17533 raft_consensus.cc:504] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20fed7f68964400a84869acd9962a256" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 40851 } } peers { permanent_uuid: "2829b5cb964d43d3b75985dfa7b19563" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 38455 } } peers { permanent_uuid: "f41dc769141846939903d45745819f86" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37197 } }
I20260806 03:41:31.345284 17533 leader_election.cc:288] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [CANDIDATE]: Term 1 election: Requested vote from peers 2829b5cb964d43d3b75985dfa7b19563 (127.15.252.67:38455), f41dc769141846939903d45745819f86 (127.15.252.66:37197)
I20260806 03:41:31.346267 17485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca885b78222414284c7dd450e6c01e1" candidate_uuid: "20fed7f68964400a84869acd9962a256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2829b5cb964d43d3b75985dfa7b19563"
I20260806 03:41:31.346290 17408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca885b78222414284c7dd450e6c01e1" candidate_uuid: "20fed7f68964400a84869acd9962a256" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f41dc769141846939903d45745819f86"
I20260806 03:41:31.347029 17485 raft_consensus.cc:3037] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:31.347054 17408 raft_consensus.cc:3037] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:31.352120 17485 raft_consensus.cc:2454] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20fed7f68964400a84869acd9962a256 in term 1.
I20260806 03:41:31.353155 17294 leader_election.cc:302] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20fed7f68964400a84869acd9962a256, 2829b5cb964d43d3b75985dfa7b19563; no voters:
I20260806 03:41:31.353327 17408 raft_consensus.cc:2454] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20fed7f68964400a84869acd9962a256 in term 1.
I20260806 03:41:31.353861 17533 raft_consensus.cc:2781] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:31.355041 17533 raft_consensus.cc:686] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [term 1 LEADER]: Becoming Leader. State: Replica: 20fed7f68964400a84869acd9962a256, State: Running, Role: LEADER
I20260806 03:41:31.355835 17533 consensus_queue.cc:237] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20fed7f68964400a84869acd9962a256" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 40851 } } peers { permanent_uuid: "2829b5cb964d43d3b75985dfa7b19563" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 38455 } } peers { permanent_uuid: "f41dc769141846939903d45745819f86" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37197 } }
I20260806 03:41:31.362409 17226 catalog_manager.cc:5495] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20fed7f68964400a84869acd9962a256 (127.15.252.65). New cstate: current_term: 1 leader_uuid: "20fed7f68964400a84869acd9962a256" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20fed7f68964400a84869acd9962a256" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 40851 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2829b5cb964d43d3b75985dfa7b19563" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 38455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f41dc769141846939903d45745819f86" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37197 } health_report { overall_health: UNKNOWN } } }
I20260806 03:41:31.866418 17542 consensus_queue.cc:1035] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2829b5cb964d43d3b75985dfa7b19563" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 38455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 03:41:31.879067 17548 consensus_queue.cc:1035] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41dc769141846939903d45745819f86" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:41:32.521543 16369 tablet_replica.cc:330] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256: stopping tablet replica
I20260806 03:41:32.522116 16369 raft_consensus.cc:2229] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:41:32.522955 16369 raft_consensus.cc:2258] T dca885b78222414284c7dd450e6c01e1 P 20fed7f68964400a84869acd9962a256 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 03:41:32.728165 17430 debug-util.cc:398] Leaking SignalData structure 0x7b08000dc3e0 after lost signal to thread 16372
W20260806 03:41:32.729009 17430 debug-util.cc:398] Leaking SignalData structure 0x7b08000dcb40 after lost signal to thread 17261
W20260806 03:41:32.729779 17430 debug-util.cc:398] Leaking SignalData structure 0x7b080019f1c0 after lost signal to thread 17356
W20260806 03:41:32.730485 17430 debug-util.cc:398] Leaking SignalData structure 0x7b080019fa00 after lost signal to thread 17433
W20260806 03:41:32.731174 17430 debug-util.cc:398] Leaking SignalData structure 0x7b080019d580 after lost signal to thread 17509
I20260806 03:41:33.760408 17553 raft_consensus.cc:482] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20fed7f68964400a84869acd9962a256)
I20260806 03:41:33.760910 17553 raft_consensus.cc:504] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20fed7f68964400a84869acd9962a256" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 40851 } } peers { permanent_uuid: "2829b5cb964d43d3b75985dfa7b19563" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 38455 } } peers { permanent_uuid: "f41dc769141846939903d45745819f86" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37197 } }
I20260806 03:41:33.762902 17553 leader_election.cc:288] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20fed7f68964400a84869acd9962a256 (127.15.252.65:40851), 2829b5cb964d43d3b75985dfa7b19563 (127.15.252.67:38455)
I20260806 03:41:33.791394 17330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca885b78222414284c7dd450e6c01e1" candidate_uuid: "f41dc769141846939903d45745819f86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20fed7f68964400a84869acd9962a256" is_pre_election: true
I20260806 03:41:33.791555 17485 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca885b78222414284c7dd450e6c01e1" candidate_uuid: "f41dc769141846939903d45745819f86" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "2829b5cb964d43d3b75985dfa7b19563" is_pre_election: true
W20260806 03:41:33.793133 17371 leader_election.cc:341] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20fed7f68964400a84869acd9962a256 (127.15.252.65:40851): Illegal state: cannot vote while shut down
I20260806 03:41:33.794209 17372 leader_election.cc:302] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f41dc769141846939903d45745819f86; no voters: 20fed7f68964400a84869acd9962a256, 2829b5cb964d43d3b75985dfa7b19563
I20260806 03:41:33.795267 17553 raft_consensus.cc:2726] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:41:33.978256 17558 raft_consensus.cc:482] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20fed7f68964400a84869acd9962a256)
I20260806 03:41:33.978847 17558 raft_consensus.cc:504] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20fed7f68964400a84869acd9962a256" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 40851 } } peers { permanent_uuid: "2829b5cb964d43d3b75985dfa7b19563" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 38455 } } peers { permanent_uuid: "f41dc769141846939903d45745819f86" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37197 } }
I20260806 03:41:33.981313 17558 leader_election.cc:288] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20fed7f68964400a84869acd9962a256 (127.15.252.65:40851), f41dc769141846939903d45745819f86 (127.15.252.66:37197)
I20260806 03:41:34.002359 17330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca885b78222414284c7dd450e6c01e1" candidate_uuid: "2829b5cb964d43d3b75985dfa7b19563" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20fed7f68964400a84869acd9962a256" is_pre_election: true
W20260806 03:41:34.004418 17448 leader_election.cc:341] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 20fed7f68964400a84869acd9962a256 (127.15.252.65:40851): Illegal state: cannot vote while shut down
I20260806 03:41:34.011201 17408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca885b78222414284c7dd450e6c01e1" candidate_uuid: "2829b5cb964d43d3b75985dfa7b19563" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f41dc769141846939903d45745819f86" is_pre_election: true
I20260806 03:41:34.011811 17408 raft_consensus.cc:2454] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2829b5cb964d43d3b75985dfa7b19563 in term 1.
I20260806 03:41:34.012892 17447 leader_election.cc:302] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2829b5cb964d43d3b75985dfa7b19563, f41dc769141846939903d45745819f86; no voters: 20fed7f68964400a84869acd9962a256
I20260806 03:41:34.013634 17558 raft_consensus.cc:2781] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 03:41:34.014102 17558 raft_consensus.cc:482] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 20fed7f68964400a84869acd9962a256)
I20260806 03:41:34.014531 17558 raft_consensus.cc:3037] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:41:34.023831 17558 raft_consensus.cc:504] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20fed7f68964400a84869acd9962a256" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 40851 } } peers { permanent_uuid: "2829b5cb964d43d3b75985dfa7b19563" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 38455 } } peers { permanent_uuid: "f41dc769141846939903d45745819f86" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37197 } }
I20260806 03:41:34.026245 17558 leader_election.cc:288] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [CANDIDATE]: Term 2 election: Requested vote from peers 20fed7f68964400a84869acd9962a256 (127.15.252.65:40851), f41dc769141846939903d45745819f86 (127.15.252.66:37197)
I20260806 03:41:34.027439 17408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca885b78222414284c7dd450e6c01e1" candidate_uuid: "2829b5cb964d43d3b75985dfa7b19563" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f41dc769141846939903d45745819f86"
I20260806 03:41:34.027547 17330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dca885b78222414284c7dd450e6c01e1" candidate_uuid: "2829b5cb964d43d3b75985dfa7b19563" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20fed7f68964400a84869acd9962a256"
I20260806 03:41:34.027990 17408 raft_consensus.cc:3037] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86 [term 1 FOLLOWER]: Advancing to term 2
W20260806 03:41:34.028971 17448 leader_election.cc:341] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 20fed7f68964400a84869acd9962a256 (127.15.252.65:40851): Illegal state: cannot vote while shut down
I20260806 03:41:34.033929 17408 raft_consensus.cc:2454] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2829b5cb964d43d3b75985dfa7b19563 in term 2.
I20260806 03:41:34.035400 17447 leader_election.cc:302] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2829b5cb964d43d3b75985dfa7b19563, f41dc769141846939903d45745819f86; no voters: 20fed7f68964400a84869acd9962a256
I20260806 03:41:34.036228 17558 raft_consensus.cc:2781] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 03:41:34.038076 17558 raft_consensus.cc:686] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [term 2 LEADER]: Becoming Leader. State: Replica: 2829b5cb964d43d3b75985dfa7b19563, State: Running, Role: LEADER
I20260806 03:41:34.039207 17558 consensus_queue.cc:237] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "20fed7f68964400a84869acd9962a256" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 40851 } } peers { permanent_uuid: "2829b5cb964d43d3b75985dfa7b19563" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 38455 } } peers { permanent_uuid: "f41dc769141846939903d45745819f86" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37197 } }
I20260806 03:41:34.571406 17448 consensus_queue.cc:579] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [LEADER]: The tablet replica hosted on peer 20fed7f68964400a84869acd9962a256 has failed
W20260806 03:41:34.572089 17448 consensus_peers.cc:487] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 -> Peer 20fed7f68964400a84869acd9962a256 (127.15.252.65:40851): Couldn't send request to peer 20fed7f68964400a84869acd9962a256. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260806 03:41:34.605494 17408 raft_consensus.cc:1261] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86 [term 2 FOLLOWER]: Refusing update from remote peer 2829b5cb964d43d3b75985dfa7b19563: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260806 03:41:34.607048 17564 consensus_queue.cc:1035] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f41dc769141846939903d45745819f86" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260806 03:41:35.560912 16369 tablet_server.cc:178] TabletServer@127.15.252.65:0 shutting down...
I20260806 03:41:35.585357 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:35.604295 16369 tablet_server.cc:194] TabletServer@127.15.252.65:0 shutdown complete.
I20260806 03:41:35.615196 16369 tablet_server.cc:178] TabletServer@127.15.252.66:0 shutting down...
I20260806 03:41:35.638827 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:35.639747 16369 tablet_replica.cc:330] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86: stopping tablet replica
I20260806 03:41:35.640588 16369 raft_consensus.cc:2229] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260806 03:41:35.641108 16369 raft_consensus.cc:2258] T dca885b78222414284c7dd450e6c01e1 P f41dc769141846939903d45745819f86 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:35.682150 16369 tablet_server.cc:194] TabletServer@127.15.252.66:0 shutdown complete.
I20260806 03:41:35.691768 16369 tablet_server.cc:178] TabletServer@127.15.252.67:0 shutting down...
I20260806 03:41:35.716197 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:35.716851 16369 tablet_replica.cc:330] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563: stopping tablet replica
I20260806 03:41:35.717391 16369 raft_consensus.cc:2229] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [term 2 LEADER]: Raft consensus shutting down.
I20260806 03:41:35.718161 16369 raft_consensus.cc:2258] T dca885b78222414284c7dd450e6c01e1 P 2829b5cb964d43d3b75985dfa7b19563 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:35.737882 16369 tablet_server.cc:194] TabletServer@127.15.252.67:0 shutdown complete.
I20260806 03:41:35.747071 16369 master.cc:537] Master@127.15.252.126:36697 shutting down...
I20260806 03:41:35.765625 16369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:41:35.766137 16369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:35.766398 16369 tablet_replica.cc:330] T 00000000000000000000000000000000 P 006341462e54415c9779495e19addadd: stopping tablet replica
I20260806 03:41:35.785033 16369 master.cc:552] Master@127.15.252.126:36697 shutdown complete.
[ OK ] TsTabletManagerITest.ReportOnReplicaHealthStatus (5628 ms)
[ RUN ] TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters
I20260806 03:41:35.812696 16369 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.252.126:39615
I20260806 03:41:35.813797 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:35.819442 17574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:35.821574 17575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:35.823583 17577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:35.824025 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:35.824843 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:35.825011 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:35.825184 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987695825172 us; error 0 us; skew 500 ppm
I20260806 03:41:35.825752 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:35.835220 16369 webserver.cc:460] Webserver started at http://127.15.252.126:32795/ using document root <none> and password file <none>
I20260806 03:41:35.835745 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:35.835929 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:35.836150 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:35.837220 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1785987678583700-16369-0/minicluster-data/master-0-root/instance:
uuid: "912cf0f9ee2f4601a81dfdddb9ef574e"
format_stamp: "Formatted at 2026-08-06 03:41:35 on dist-test-slave-csg5"
I20260806 03:41:35.841997 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 03:41:35.846079 17582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:35.847082 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 03:41:35.847410 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1785987678583700-16369-0/minicluster-data/master-0-root
uuid: "912cf0f9ee2f4601a81dfdddb9ef574e"
format_stamp: "Formatted at 2026-08-06 03:41:35 on dist-test-slave-csg5"
I20260806 03:41:35.847764 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1785987678583700-16369-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1785987678583700-16369-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1785987678583700-16369-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:35.870409 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:35.871526 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:35.912972 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.126:39615
I20260806 03:41:35.913108 17633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.126:39615 every 8 connection(s)
I20260806 03:41:35.917133 17634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:35.928279 17634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e: Bootstrap starting.
I20260806 03:41:35.932816 17634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:35.937580 17634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e: No bootstrap required, opened a new log
I20260806 03:41:35.939627 17634 raft_consensus.cc:348] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912cf0f9ee2f4601a81dfdddb9ef574e" member_type: VOTER }
I20260806 03:41:35.940025 17634 raft_consensus.cc:374] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:35.940299 17634 raft_consensus.cc:729] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 912cf0f9ee2f4601a81dfdddb9ef574e, State: Initialized, Role: FOLLOWER
I20260806 03:41:35.940909 17634 consensus_queue.cc:260] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912cf0f9ee2f4601a81dfdddb9ef574e" member_type: VOTER }
I20260806 03:41:35.941347 17634 raft_consensus.cc:388] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:41:35.941601 17634 raft_consensus.cc:482] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:41:35.941875 17634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:35.947141 17634 raft_consensus.cc:504] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912cf0f9ee2f4601a81dfdddb9ef574e" member_type: VOTER }
I20260806 03:41:35.947875 17634 leader_election.cc:302] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 912cf0f9ee2f4601a81dfdddb9ef574e; no voters:
I20260806 03:41:35.949517 17634 leader_election.cc:288] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:41:35.950100 17637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:35.960639 17637 raft_consensus.cc:686] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e [term 1 LEADER]: Becoming Leader. State: Replica: 912cf0f9ee2f4601a81dfdddb9ef574e, State: Running, Role: LEADER
I20260806 03:41:35.961401 17634 sys_catalog.cc:564] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:41:35.961529 17637 consensus_queue.cc:237] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912cf0f9ee2f4601a81dfdddb9ef574e" member_type: VOTER }
I20260806 03:41:35.967660 17638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "912cf0f9ee2f4601a81dfdddb9ef574e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912cf0f9ee2f4601a81dfdddb9ef574e" member_type: VOTER } }
I20260806 03:41:35.967871 17639 sys_catalog.cc:455] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 912cf0f9ee2f4601a81dfdddb9ef574e. Latest consensus state: current_term: 1 leader_uuid: "912cf0f9ee2f4601a81dfdddb9ef574e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "912cf0f9ee2f4601a81dfdddb9ef574e" member_type: VOTER } }
I20260806 03:41:35.968588 17639 sys_catalog.cc:458] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:35.968621 17638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:35.973773 17646 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:41:35.978593 17646 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:41:35.980506 16369 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260806 03:41:35.987888 17646 catalog_manager.cc:1347] Generated new cluster ID: d5cea77cea7a4fab82dddca5eba66161
I20260806 03:41:35.988217 17646 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:41:36.006587 17646 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:41:36.008520 17646 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:41:36.027092 17646 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e: Generated new TSK 0
I20260806 03:41:36.028026 17646 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:41:36.047171 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:36.054329 17655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:36.055086 17656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:36.057310 17658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:36.057756 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:36.058645 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:36.058925 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:36.059096 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987696059058 us; error 0 us; skew 500 ppm
I20260806 03:41:36.059650 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:36.062392 16369 webserver.cc:460] Webserver started at http://127.15.252.65:44801/ using document root <none> and password file <none>
I20260806 03:41:36.063005 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:36.063205 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:36.063503 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:36.064599 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1785987678583700-16369-0/minicluster-data/ts-0-root/instance:
uuid: "2b3e55e782c24eb68aade5e2deebbfcf"
format_stamp: "Formatted at 2026-08-06 03:41:36 on dist-test-slave-csg5"
I20260806 03:41:36.069584 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 03:41:36.073644 17664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:36.074888 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 03:41:36.075243 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1785987678583700-16369-0/minicluster-data/ts-0-root
uuid: "2b3e55e782c24eb68aade5e2deebbfcf"
format_stamp: "Formatted at 2026-08-06 03:41:36 on dist-test-slave-csg5"
I20260806 03:41:36.075596 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1785987678583700-16369-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1785987678583700-16369-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1785987678583700-16369-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:36.100807 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:36.102370 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:36.104219 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:36.106956 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:36.107203 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:36.107494 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:36.107682 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:36.153514 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.65:33205
I20260806 03:41:36.153605 17727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.65:33205 every 8 connection(s)
I20260806 03:41:36.172254 17728 heartbeater.cc:347] Connected to a master server at 127.15.252.126:39615
I20260806 03:41:36.172783 17728 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:36.173547 17728 heartbeater.cc:511] Master 127.15.252.126:39615 requested a full tablet report, sending...
I20260806 03:41:36.175555 17599 ts_manager.cc:194] Registered new tserver with Master: 2b3e55e782c24eb68aade5e2deebbfcf (127.15.252.65:33205)
I20260806 03:41:36.176213 16369 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.019166387s
I20260806 03:41:36.178179 17599 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60798
I20260806 03:41:36.202139 16369 tablet_server.cc:178] TabletServer@127.15.252.65:0 shutting down...
I20260806 03:41:36.221859 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:36.239493 16369 tablet_server.cc:194] TabletServer@127.15.252.65:0 shutdown complete.
I20260806 03:41:36.248720 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:41:36.251462 16369 tablet_server.cc:93] deduplicated master addresses: 127.15.252.126:39615
W20260806 03:41:36.256243 17738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:36.256477 17739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:36.258441 17741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:36.258591 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:36.259740 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:36.259935 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:36.260141 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987696260126 us; error 0 us; skew 500 ppm
I20260806 03:41:36.260726 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:36.263590 16369 webserver.cc:460] Webserver started at http://127.15.252.65:44801/ using document root <none> and password file <none>
I20260806 03:41:36.264132 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:36.264310 16369 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260806 03:41:36.268469 16369 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 03:41:36.272626 17746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:36.273713 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 03:41:36.274026 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1785987678583700-16369-0/minicluster-data/ts-0-root
uuid: "2b3e55e782c24eb68aade5e2deebbfcf"
format_stamp: "Formatted at 2026-08-06 03:41:36 on dist-test-slave-csg5"
I20260806 03:41:36.274331 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1785987678583700-16369-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1785987678583700-16369-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters.1785987678583700-16369-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:36.298044 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:36.299226 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:36.300752 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:36.303431 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:36.303648 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:36.303988 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:36.304136 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:36.349774 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.65:33205
I20260806 03:41:36.349707 17809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.65:33205 every 8 connection(s)
I20260806 03:41:36.359769 16369 tablet_server.cc:178] TabletServer@127.15.252.65:33205 shutting down...
I20260806 03:41:36.390108 17810 heartbeater.cc:347] Connected to a master server at 127.15.252.126:39615
I20260806 03:41:36.390618 17810 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:36.391379 17810 heartbeater.cc:511] Master 127.15.252.126:39615 requested a full tablet report, sending...
I20260806 03:41:36.393925 17599 ts_manager.cc:194] Re-registered known tserver with Master: 2b3e55e782c24eb68aade5e2deebbfcf (127.15.252.65:33205)
I20260806 03:41:36.395741 17599 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60810
I20260806 03:41:36.399050 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:36.416934 16369 tablet_server.cc:194] TabletServer@127.15.252.65:33205 shutdown complete.
I20260806 03:41:36.425827 16369 master.cc:537] Master@127.15.252.126:39615 shutting down...
I20260806 03:41:36.444108 16369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:41:36.444703 16369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:36.445179 16369 tablet_replica.cc:330] T 00000000000000000000000000000000 P 912cf0f9ee2f4601a81dfdddb9ef574e: stopping tablet replica
I20260806 03:41:36.465758 16369 master.cc:552] Master@127.15.252.126:39615 shutdown complete.
[ OK ] TsTabletManagerITest.TestDeduplicateMasterAddrsForHeartbeaters (677 ms)
[ RUN ] TsTabletManagerITest.TestTableStats
I20260806 03:41:36.490362 16369 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.252.126:42845,127.15.252.125:34597,127.15.252.124:39623
I20260806 03:41:36.491688 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:36.496869 17817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:36.498075 17818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:36.500223 17820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:36.500248 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:36.501257 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:36.501466 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:36.501629 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987696501613 us; error 0 us; skew 500 ppm
I20260806 03:41:36.502182 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:36.504807 16369 webserver.cc:460] Webserver started at http://127.15.252.126:35243/ using document root <none> and password file <none>
I20260806 03:41:36.505323 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:36.505532 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:36.505795 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:36.507015 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-0-root/instance:
uuid: "30063023a8844495837679314257b563"
format_stamp: "Formatted at 2026-08-06 03:41:36 on dist-test-slave-csg5"
I20260806 03:41:36.511829 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 03:41:36.515586 17825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:36.516484 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 03:41:36.516795 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-0-root
uuid: "30063023a8844495837679314257b563"
format_stamp: "Formatted at 2026-08-06 03:41:36 on dist-test-slave-csg5"
I20260806 03:41:36.517112 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:36.526263 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:36.527396 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:36.568538 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.126:42845
I20260806 03:41:36.568624 17877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.126:42845 every 8 connection(s)
I20260806 03:41:36.572722 17878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:36.572942 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:36.579063 17880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:36.579728 17881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:36.583704 16369 server_base.cc:1034] running on GCE node
W20260806 03:41:36.583838 17883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:36.583963 17878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:41:36.585073 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:36.585314 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:36.585558 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987696585540 us; error 0 us; skew 500 ppm
I20260806 03:41:36.586426 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:36.590102 16369 webserver.cc:460] Webserver started at http://127.15.252.125:37953/ using document root <none> and password file <none>
I20260806 03:41:36.590780 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:36.591077 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:36.591460 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:36.593123 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-1-root/instance:
uuid: "2a5ff5ca39d348fa837104d428536561"
format_stamp: "Formatted at 2026-08-06 03:41:36 on dist-test-slave-csg5"
I20260806 03:41:36.599261 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260806 03:41:36.603935 17890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:36.604971 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260806 03:41:36.605175 17878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:41:36.605341 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-1-root
uuid: "2a5ff5ca39d348fa837104d428536561"
format_stamp: "Formatted at 2026-08-06 03:41:36 on dist-test-slave-csg5"
I20260806 03:41:36.605682 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260806 03:41:36.607877 17829 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.252.125:34597: connect: Connection refused (error 111)
W20260806 03:41:36.611980 17878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.252.125:34597: Network error: Client connection negotiation failed: client connection to 127.15.252.125:34597: connect: Connection refused (error 111)
I20260806 03:41:36.620052 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:36.621217 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:36.650614 17878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } attempt: 1
W20260806 03:41:36.655262 17878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.252.125:34597: Network error: Client connection negotiation failed: client connection to 127.15.252.125:34597: connect: Connection refused (error 111)
I20260806 03:41:36.663808 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.125:34597
I20260806 03:41:36.663937 17942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.125:34597 every 8 connection(s)
I20260806 03:41:36.668206 17943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:36.668634 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:36.676610 17945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:36.678689 17943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } has no permanent_uuid. Determining permanent_uuid...
W20260806 03:41:36.680130 17946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:36.682545 16369 server_base.cc:1034] running on GCE node
W20260806 03:41:36.683439 17948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:36.684500 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:36.684937 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:36.685176 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987696685162 us; error 0 us; skew 500 ppm
I20260806 03:41:36.685753 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:36.688827 16369 webserver.cc:460] Webserver started at http://127.15.252.124:41187/ using document root <none> and password file <none>
I20260806 03:41:36.689412 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:36.689651 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:36.689960 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:36.691293 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-2-root/instance:
uuid: "c0591ed664d442fdaebaa2cd64c84b9f"
format_stamp: "Formatted at 2026-08-06 03:41:36 on dist-test-slave-csg5"
I20260806 03:41:36.695767 17878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } attempt: 2
I20260806 03:41:36.695776 17943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:41:36.696931 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 03:41:36.702270 17956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:36.703373 16369 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260806 03:41:36.703748 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-2-root
uuid: "c0591ed664d442fdaebaa2cd64c84b9f"
format_stamp: "Formatted at 2026-08-06 03:41:36 on dist-test-slave-csg5"
I20260806 03:41:36.704167 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:36.719235 17943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:41:36.719651 17878 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } has no permanent_uuid. Determining permanent_uuid...
W20260806 03:41:36.724858 17878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.252.124:39623: Network error: Client connection negotiation failed: client connection to 127.15.252.124:39623: connect: Connection refused (error 111)
W20260806 03:41:36.726194 17943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.252.124:39623: Network error: Client connection negotiation failed: client connection to 127.15.252.124:39623: connect: Connection refused (error 111)
I20260806 03:41:36.757175 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:36.758347 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:36.759851 17943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } attempt: 1
I20260806 03:41:36.760313 17878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } attempt: 1
W20260806 03:41:36.764812 17878 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.252.124:39623: Network error: Client connection negotiation failed: client connection to 127.15.252.124:39623: connect: Connection refused (error 111)
W20260806 03:41:36.765733 17943 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.252.124:39623: Network error: Client connection negotiation failed: client connection to 127.15.252.124:39623: connect: Connection refused (error 111)
I20260806 03:41:36.799901 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.124:39623
I20260806 03:41:36.799989 18010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.124:39623 every 8 connection(s)
I20260806 03:41:36.803032 16369 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260806 03:41:36.804139 18011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:36.804378 17943 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } attempt: 2
I20260806 03:41:36.810922 18011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:41:36.825363 17943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561: Bootstrap starting.
I20260806 03:41:36.826112 18011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:41:36.831028 17943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:36.836266 17943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561: No bootstrap required, opened a new log
I20260806 03:41:36.838953 17943 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:36.839773 17943 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:36.840085 17943 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a5ff5ca39d348fa837104d428536561, State: Initialized, Role: FOLLOWER
I20260806 03:41:36.840301 17878 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } attempt: 2
I20260806 03:41:36.840955 18011 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:41:36.840813 17943 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:36.845319 18016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:36.846813 18016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:36.846807 17943 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:41:36.866228 17878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563: Bootstrap starting.
W20260806 03:41:36.870016 18028 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:41:36.870365 18028 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:41:36.872885 17878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:36.878528 17878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563: No bootstrap required, opened a new log
I20260806 03:41:36.879865 18011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f: Bootstrap starting.
I20260806 03:41:36.881531 17878 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:36.882414 17878 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:36.882772 17878 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30063023a8844495837679314257b563, State: Initialized, Role: FOLLOWER
I20260806 03:41:36.883561 17878 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:36.885891 18030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:36.886727 18011 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:36.886714 18030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:36.887852 17878 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:41:36.893024 18011 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f: No bootstrap required, opened a new log
I20260806 03:41:36.894526 18016 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:41:36.895102 18016 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:36.897061 18016 leader_election.cc:288] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30063023a8844495837679314257b563 (127.15.252.126:42845), c0591ed664d442fdaebaa2cd64c84b9f (127.15.252.124:39623)
I20260806 03:41:36.896437 18011 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:36.898106 17853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a5ff5ca39d348fa837104d428536561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30063023a8844495837679314257b563" is_pre_election: true
I20260806 03:41:36.898933 17853 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a5ff5ca39d348fa837104d428536561 in term 0.
I20260806 03:41:36.898175 18011 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:36.899937 18011 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0591ed664d442fdaebaa2cd64c84b9f, State: Initialized, Role: FOLLOWER
I20260806 03:41:36.900579 17893 leader_election.cc:302] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a5ff5ca39d348fa837104d428536561, 30063023a8844495837679314257b563; no voters:
I20260806 03:41:36.900960 18011 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:36.901844 18016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:41:36.903110 18016 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:41:36.903657 18016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:36.910701 18041 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:36.911743 18041 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:36.913488 18016 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:36.914597 18011 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:41:36.914644 17985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a5ff5ca39d348fa837104d428536561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" is_pre_election: true
I20260806 03:41:36.915800 17985 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a5ff5ca39d348fa837104d428536561 in term 0.
I20260806 03:41:36.917269 17853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a5ff5ca39d348fa837104d428536561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30063023a8844495837679314257b563"
I20260806 03:41:36.917629 18016 leader_election.cc:288] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [CANDIDATE]: Term 1 election: Requested vote from peers 30063023a8844495837679314257b563 (127.15.252.126:42845), c0591ed664d442fdaebaa2cd64c84b9f (127.15.252.124:39623)
I20260806 03:41:36.917944 17853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:36.918988 17984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a5ff5ca39d348fa837104d428536561" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0591ed664d442fdaebaa2cd64c84b9f"
I20260806 03:41:36.919760 17984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:36.925908 17853 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a5ff5ca39d348fa837104d428536561 in term 1.
I20260806 03:41:36.927635 17984 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a5ff5ca39d348fa837104d428536561 in term 1.
I20260806 03:41:36.927564 17893 leader_election.cc:302] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a5ff5ca39d348fa837104d428536561, 30063023a8844495837679314257b563; no voters:
I20260806 03:41:36.928598 18016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:36.930052 18016 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 1 LEADER]: Becoming Leader. State: Replica: 2a5ff5ca39d348fa837104d428536561, State: Running, Role: LEADER
I20260806 03:41:36.931062 18016 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:36.935099 18044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a5ff5ca39d348fa837104d428536561. Latest consensus state: current_term: 1 leader_uuid: "2a5ff5ca39d348fa837104d428536561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:36.935946 18044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:36.938025 16369 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20260806 03:41:36.938360 16369 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20260806 03:41:36.939821 18049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260806 03:41:36.942126 18045 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:41:36.942428 18045 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:41:36.948688 18049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:41:36.949990 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:36.952504 18058 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:41:36.952839 18058 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:41:36.959836 17984 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 1 FOLLOWER]: Refusing update from remote peer 2a5ff5ca39d348fa837104d428536561: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:41:36.959836 17853 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 1 FOLLOWER]: Refusing update from remote peer 2a5ff5ca39d348fa837104d428536561: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260806 03:41:36.961581 18061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:36.962173 18016 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 03:41:36.963295 18063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:36.963932 18060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:36.964308 18016 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:41:36.972074 18041 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a5ff5ca39d348fa837104d428536561. Latest consensus state: current_term: 1 leader_uuid: "2a5ff5ca39d348fa837104d428536561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:36.972898 18041 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:36.973785 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:36.974714 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:36.974965 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:36.975148 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987696975135 us; error 0 us; skew 500 ppm
I20260806 03:41:36.975713 18059 mvcc.cc:205] Tried to move back new op lower bound from 7315405606727798784 to 7315405606637551616. Current Snapshot: MvccSnapshot[applied={T|T < 7315405606727798784}]
I20260806 03:41:36.976519 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:36.977720 18030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a5ff5ca39d348fa837104d428536561. Latest consensus state: current_term: 1 leader_uuid: "2a5ff5ca39d348fa837104d428536561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:36.978749 18030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:36.981817 18041 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2a5ff5ca39d348fa837104d428536561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:36.982605 18041 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:36.984863 18049 catalog_manager.cc:1347] Generated new cluster ID: 8225191a001148569992571dec22b329
I20260806 03:41:36.985272 18049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:41:36.989885 18030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2a5ff5ca39d348fa837104d428536561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:36.990970 18030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:36.995792 18044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2a5ff5ca39d348fa837104d428536561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:36.995975 18016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2a5ff5ca39d348fa837104d428536561" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:36.996783 18016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:36.997254 18044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:36.997841 16369 webserver.cc:460] Webserver started at http://127.15.252.65:45261/ using document root <none> and password file <none>
I20260806 03:41:36.998479 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:36.998741 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:36.999049 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:37.000706 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-0-root/instance:
uuid: "4abe04524a7046018757fcdfd13b815d"
format_stamp: "Formatted at 2026-08-06 03:41:36 on dist-test-slave-csg5"
I20260806 03:41:37.007164 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.005s
I20260806 03:41:37.014511 18076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:37.016322 16369 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.003s
I20260806 03:41:37.016801 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-0-root
uuid: "4abe04524a7046018757fcdfd13b815d"
format_stamp: "Formatted at 2026-08-06 03:41:36 on dist-test-slave-csg5"
I20260806 03:41:37.017246 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:37.024410 18049 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:41:37.026798 18049 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:41:37.031549 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:37.032850 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:37.034827 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:37.037868 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:37.038089 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:37.038429 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:37.038617 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:37.062616 18049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561: Generated new TSK 0
I20260806 03:41:37.063772 18049 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:41:37.099892 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.65:35231
I20260806 03:41:37.099987 18139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.65:35231 every 8 connection(s)
I20260806 03:41:37.116350 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:37.128826 18147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:37.129944 18141 heartbeater.cc:347] Connected to a master server at 127.15.252.125:34597
I20260806 03:41:37.130729 18141 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:37.131983 18141 heartbeater.cc:511] Master 127.15.252.125:34597 requested a full tablet report, sending...
I20260806 03:41:37.134625 18140 heartbeater.cc:347] Connected to a master server at 127.15.252.124:39623
I20260806 03:41:37.135041 18140 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:37.135756 17908 ts_manager.cc:194] Registered new tserver with Master: 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231)
I20260806 03:41:37.136009 18140 heartbeater.cc:511] Master 127.15.252.124:39623 requested a full tablet report, sending...
I20260806 03:41:37.139648 17974 ts_manager.cc:194] Registered new tserver with Master: 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231)
I20260806 03:41:37.139815 17908 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53836
I20260806 03:41:37.143461 18142 heartbeater.cc:347] Connected to a master server at 127.15.252.126:42845
I20260806 03:41:37.143657 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:37.143895 18142 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:37.144796 18142 heartbeater.cc:511] Master 127.15.252.126:42845 requested a full tablet report, sending...
W20260806 03:41:37.145032 18148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:37.145893 18151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:37.147339 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:37.147786 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:37.148042 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987697148025 us; error 0 us; skew 500 ppm
I20260806 03:41:37.148121 17843 ts_manager.cc:194] Registered new tserver with Master: 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231)
I20260806 03:41:37.148883 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:37.151736 16369 webserver.cc:460] Webserver started at http://127.15.252.66:34677/ using document root <none> and password file <none>
I20260806 03:41:37.152215 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:37.152433 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:37.152705 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:37.153784 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-1-root/instance:
uuid: "491cdd4ce26949848b317caa81d5279d"
format_stamp: "Formatted at 2026-08-06 03:41:37 on dist-test-slave-csg5"
I20260806 03:41:37.158285 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 03:41:37.162834 18156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:37.163631 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 03:41:37.163993 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-1-root
uuid: "491cdd4ce26949848b317caa81d5279d"
format_stamp: "Formatted at 2026-08-06 03:41:37 on dist-test-slave-csg5"
I20260806 03:41:37.164304 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:37.180274 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:37.181526 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:37.183712 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:37.186975 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:37.187202 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:37.187449 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:37.187674 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:37.244410 18141 heartbeater.cc:503] Master 127.15.252.125:34597 was elected leader, sending a full tablet report...
I20260806 03:41:37.285521 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.66:43729
I20260806 03:41:37.285660 18221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.66:43729 every 8 connection(s)
I20260806 03:41:37.308161 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:37.323271 18229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:37.330010 18223 heartbeater.cc:347] Connected to a master server at 127.15.252.125:34597
I20260806 03:41:37.330507 18223 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:37.337369 18223 heartbeater.cc:511] Master 127.15.252.125:34597 requested a full tablet report, sending...
W20260806 03:41:37.338779 18230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:37.340874 17908 ts_manager.cc:194] Registered new tserver with Master: 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:37.341579 18222 heartbeater.cc:347] Connected to a master server at 127.15.252.124:39623
I20260806 03:41:37.342065 18222 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:37.343390 18222 heartbeater.cc:511] Master 127.15.252.124:39623 requested a full tablet report, sending...
I20260806 03:41:37.345628 17908 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53848
I20260806 03:41:37.350217 17974 ts_manager.cc:194] Registered new tserver with Master: 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:37.352075 18224 heartbeater.cc:347] Connected to a master server at 127.15.252.126:42845
I20260806 03:41:37.352526 18224 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:37.355726 18224 heartbeater.cc:511] Master 127.15.252.126:42845 requested a full tablet report, sending...
I20260806 03:41:37.359146 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:37.360445 17843 ts_manager.cc:194] Registered new tserver with Master: 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
W20260806 03:41:37.360618 18232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:37.361979 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:37.362294 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:37.362532 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987697362516 us; error 0 us; skew 500 ppm
I20260806 03:41:37.363418 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:37.366529 16369 webserver.cc:460] Webserver started at http://127.15.252.67:45789/ using document root <none> and password file <none>
I20260806 03:41:37.367182 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:37.367401 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:37.367733 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:37.368813 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-2-root/instance:
uuid: "6eb7ccdcc221406296e866e74a37ed75"
format_stamp: "Formatted at 2026-08-06 03:41:37 on dist-test-slave-csg5"
I20260806 03:41:37.373497 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 03:41:37.377643 18237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:37.378996 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 03:41:37.379426 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-2-root
uuid: "6eb7ccdcc221406296e866e74a37ed75"
format_stamp: "Formatted at 2026-08-06 03:41:37 on dist-test-slave-csg5"
I20260806 03:41:37.379756 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:37.394215 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:37.395901 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:37.397397 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:37.400269 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:37.400465 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:37.400753 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:37.400945 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:37.458918 18223 heartbeater.cc:503] Master 127.15.252.125:34597 was elected leader, sending a full tablet report...
I20260806 03:41:37.523604 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.67:35363
I20260806 03:41:37.523732 18301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.67:35363 every 8 connection(s)
I20260806 03:41:37.577936 18303 heartbeater.cc:347] Connected to a master server at 127.15.252.124:39623
I20260806 03:41:37.578305 18303 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:37.579313 18303 heartbeater.cc:511] Master 127.15.252.124:39623 requested a full tablet report, sending...
I20260806 03:41:37.580127 18304 heartbeater.cc:347] Connected to a master server at 127.15.252.125:34597
I20260806 03:41:37.580575 18304 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:37.581589 18304 heartbeater.cc:511] Master 127.15.252.125:34597 requested a full tablet report, sending...
I20260806 03:41:37.584483 18305 heartbeater.cc:347] Connected to a master server at 127.15.252.126:42845
I20260806 03:41:37.584909 18305 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:37.585812 18305 heartbeater.cc:511] Master 127.15.252.126:42845 requested a full tablet report, sending...
I20260806 03:41:37.585763 17907 ts_manager.cc:194] Registered new tserver with Master: 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:37.587985 17974 ts_manager.cc:194] Registered new tserver with Master: 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:37.588563 17907 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53714
I20260806 03:41:37.588562 17843 ts_manager.cc:194] Registered new tserver with Master: 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:37.588829 16369 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.034872497s
I20260806 03:41:37.640851 17907 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53716:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20260806 03:41:37.643412 17907 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:41:37.692863 18304 heartbeater.cc:503] Master 127.15.252.125:34597 was elected leader, sending a full tablet report...
I20260806 03:41:37.749214 18266 tablet_service.cc:1467] Processing CreateTablet for tablet 4f6ce5ed631f413d907292ee85373f54 (DEFAULT_TABLE table=test-table [id=325a9a6c58fd4d258c4388b45595aefe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260806 03:41:37.750490 18265 tablet_service.cc:1467] Processing CreateTablet for tablet 5390b9a07b5f4cd8b2330fd0974c8821 (DEFAULT_TABLE table=test-table [id=325a9a6c58fd4d258c4388b45595aefe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260806 03:41:37.751085 18264 tablet_service.cc:1467] Processing CreateTablet for tablet 6c23a83f61a74931bb2a5239d32e1cad (DEFAULT_TABLE table=test-table [id=325a9a6c58fd4d258c4388b45595aefe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260806 03:41:37.751758 18266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f6ce5ed631f413d907292ee85373f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:37.755565 18265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5390b9a07b5f4cd8b2330fd0974c8821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:37.757153 18184 tablet_service.cc:1467] Processing CreateTablet for tablet 6c23a83f61a74931bb2a5239d32e1cad (DEFAULT_TABLE table=test-table [id=325a9a6c58fd4d258c4388b45595aefe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260806 03:41:37.757700 18264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c23a83f61a74931bb2a5239d32e1cad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:37.756438 18185 tablet_service.cc:1467] Processing CreateTablet for tablet 5390b9a07b5f4cd8b2330fd0974c8821 (DEFAULT_TABLE table=test-table [id=325a9a6c58fd4d258c4388b45595aefe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260806 03:41:37.758828 18184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c23a83f61a74931bb2a5239d32e1cad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:37.759774 18185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5390b9a07b5f4cd8b2330fd0974c8821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:37.756079 18186 tablet_service.cc:1467] Processing CreateTablet for tablet 4f6ce5ed631f413d907292ee85373f54 (DEFAULT_TABLE table=test-table [id=325a9a6c58fd4d258c4388b45595aefe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260806 03:41:37.764618 18186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f6ce5ed631f413d907292ee85373f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:37.778373 18105 tablet_service.cc:1467] Processing CreateTablet for tablet 4f6ce5ed631f413d907292ee85373f54 (DEFAULT_TABLE table=test-table [id=325a9a6c58fd4d258c4388b45595aefe]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20260806 03:41:37.781083 18103 tablet_service.cc:1467] Processing CreateTablet for tablet 6c23a83f61a74931bb2a5239d32e1cad (DEFAULT_TABLE table=test-table [id=325a9a6c58fd4d258c4388b45595aefe]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20260806 03:41:37.779188 18104 tablet_service.cc:1467] Processing CreateTablet for tablet 5390b9a07b5f4cd8b2330fd0974c8821 (DEFAULT_TABLE table=test-table [id=325a9a6c58fd4d258c4388b45595aefe]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20260806 03:41:37.783250 18105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f6ce5ed631f413d907292ee85373f54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:37.797878 18103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c23a83f61a74931bb2a5239d32e1cad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:37.798978 18104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5390b9a07b5f4cd8b2330fd0974c8821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:37.835404 18332 tablet_bootstrap.cc:492] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d: Bootstrap starting.
I20260806 03:41:37.853535 18334 tablet_bootstrap.cc:492] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d: Bootstrap starting.
I20260806 03:41:37.853457 18333 tablet_bootstrap.cc:492] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75: Bootstrap starting.
I20260806 03:41:37.861996 18333 tablet_bootstrap.cc:654] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:37.862046 18332 tablet_bootstrap.cc:654] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:37.865020 18334 tablet_bootstrap.cc:654] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:37.871579 18332 tablet_bootstrap.cc:492] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d: No bootstrap required, opened a new log
I20260806 03:41:37.872277 18332 ts_tablet_manager.cc:1397] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d: Time spent bootstrapping tablet: real 0.037s user 0.010s sys 0.008s
I20260806 03:41:37.876704 18333 tablet_bootstrap.cc:492] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75: No bootstrap required, opened a new log
I20260806 03:41:37.876389 18332 raft_consensus.cc:348] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:37.877270 18333 ts_tablet_manager.cc:1397] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75: Time spent bootstrapping tablet: real 0.027s user 0.020s sys 0.000s
I20260806 03:41:37.877296 18332 raft_consensus.cc:374] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:37.877848 18334 tablet_bootstrap.cc:492] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d: No bootstrap required, opened a new log
I20260806 03:41:37.877849 18332 raft_consensus.cc:729] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 491cdd4ce26949848b317caa81d5279d, State: Initialized, Role: FOLLOWER
I20260806 03:41:37.878314 18334 ts_tablet_manager.cc:1397] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d: Time spent bootstrapping tablet: real 0.025s user 0.018s sys 0.004s
I20260806 03:41:37.879359 18332 consensus_queue.cc:260] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:37.881877 18333 raft_consensus.cc:348] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:37.882622 18333 raft_consensus.cc:374] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:37.883064 18333 raft_consensus.cc:729] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eb7ccdcc221406296e866e74a37ed75, State: Initialized, Role: FOLLOWER
I20260806 03:41:37.883656 18334 raft_consensus.cc:348] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:37.883833 18333 consensus_queue.cc:260] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:37.884675 18334 raft_consensus.cc:374] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:37.885073 18334 raft_consensus.cc:729] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abe04524a7046018757fcdfd13b815d, State: Initialized, Role: FOLLOWER
I20260806 03:41:37.886477 18334 consensus_queue.cc:260] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:37.889866 18333 ts_tablet_manager.cc:1428] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75: Time spent starting tablet: real 0.012s user 0.000s sys 0.013s
I20260806 03:41:37.890133 18332 ts_tablet_manager.cc:1428] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d: Time spent starting tablet: real 0.017s user 0.016s sys 0.001s
I20260806 03:41:37.899478 18332 tablet_bootstrap.cc:492] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d: Bootstrap starting.
I20260806 03:41:37.924110 18332 tablet_bootstrap.cc:654] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:37.938001 18338 raft_consensus.cc:482] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:41:37.928066 18333 tablet_bootstrap.cc:492] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75: Bootstrap starting.
I20260806 03:41:37.939152 18338 raft_consensus.cc:504] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:37.952255 18338 leader_election.cc:288] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363), 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231)
I20260806 03:41:37.972494 18045 catalog_manager.cc:1259] Loaded cluster ID: 8225191a001148569992571dec22b329
I20260806 03:41:37.972982 18333 tablet_bootstrap.cc:654] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:37.973017 18045 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563: loading cluster ID for follower catalog manager: success
I20260806 03:41:37.975679 18334 ts_tablet_manager.cc:1428] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d: Time spent starting tablet: real 0.097s user 0.015s sys 0.043s
I20260806 03:41:37.978994 18058 catalog_manager.cc:1259] Loaded cluster ID: 8225191a001148569992571dec22b329
I20260806 03:41:37.981191 18339 raft_consensus.cc:482] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:41:37.980748 18334 tablet_bootstrap.cc:492] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d: Bootstrap starting.
I20260806 03:41:37.979976 18058 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f: loading cluster ID for follower catalog manager: success
I20260806 03:41:37.982228 18339 raft_consensus.cc:504] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:38.008394 18339 leader_election.cc:288] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:38.014420 18340 raft_consensus.cc:482] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:41:38.017912 18045 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563: acquiring CA information for follower catalog manager: success
I20260806 03:41:38.015726 18340 raft_consensus.cc:504] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:38.074132 18340 leader_election.cc:288] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:38.084088 18058 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f: acquiring CA information for follower catalog manager: success
I20260806 03:41:38.092522 18333 tablet_bootstrap.cc:492] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75: No bootstrap required, opened a new log
I20260806 03:41:38.093063 18045 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 03:41:38.094189 18334 tablet_bootstrap.cc:654] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:38.093094 18333 ts_tablet_manager.cc:1397] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75: Time spent bootstrapping tablet: real 0.166s user 0.007s sys 0.013s
I20260806 03:41:38.114549 18332 tablet_bootstrap.cc:492] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d: No bootstrap required, opened a new log
I20260806 03:41:38.120381 18332 ts_tablet_manager.cc:1397] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d: Time spent bootstrapping tablet: real 0.221s user 0.020s sys 0.027s
I20260806 03:41:38.132082 18276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
W20260806 03:41:38.144085 18159 leader_election.cc:341] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363): Illegal state: must be running to vote when last-logged opid is not known
I20260806 03:41:38.158428 18196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d" is_pre_election: true
W20260806 03:41:38.168015 18239 leader_election.cc:341] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729): Illegal state: must be running to vote when last-logged opid is not known
I20260806 03:41:38.171290 18333 raft_consensus.cc:348] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:38.172595 18333 raft_consensus.cc:374] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:38.173213 18333 raft_consensus.cc:729] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eb7ccdcc221406296e866e74a37ed75, State: Initialized, Role: FOLLOWER
I20260806 03:41:38.175215 18058 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 03:41:38.173957 18332 raft_consensus.cc:348] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:38.188233 18332 raft_consensus.cc:374] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:38.188645 18332 raft_consensus.cc:729] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 491cdd4ce26949848b317caa81d5279d, State: Initialized, Role: FOLLOWER
I20260806 03:41:38.191602 18332 consensus_queue.cc:260] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:38.191860 18333 consensus_queue.cc:260] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:38.195372 18334 tablet_bootstrap.cc:492] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d: No bootstrap required, opened a new log
I20260806 03:41:38.196209 18334 ts_tablet_manager.cc:1397] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d: Time spent bootstrapping tablet: real 0.216s user 0.014s sys 0.017s
I20260806 03:41:38.201915 18334 raft_consensus.cc:348] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:38.202826 18334 raft_consensus.cc:374] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:38.203235 18334 raft_consensus.cc:729] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abe04524a7046018757fcdfd13b815d, State: Initialized, Role: FOLLOWER
I20260806 03:41:38.203580 18339 raft_consensus.cc:482] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:41:38.204078 18339 raft_consensus.cc:504] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:38.207159 18115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:38.203990 18334 consensus_queue.cc:260] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:38.210001 18114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:38.210775 18114 raft_consensus.cc:2454] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6eb7ccdcc221406296e866e74a37ed75 in term 0.
I20260806 03:41:38.210906 18113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:38.212328 18241 leader_election.cc:302] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75; no voters:
W20260806 03:41:38.212812 18160 leader_election.cc:341] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231): Illegal state: must be running to vote when last-logged opid is not known
I20260806 03:41:38.213403 18160 leader_election.cc:302] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d; no voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75
I20260806 03:41:38.214107 18115 raft_consensus.cc:2454] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6eb7ccdcc221406296e866e74a37ed75 in term 0.
I20260806 03:41:38.214722 18333 ts_tablet_manager.cc:1428] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75: Time spent starting tablet: real 0.106s user 0.009s sys 0.002s
I20260806 03:41:38.215451 18341 raft_consensus.cc:2726] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 03:41:38.215229 18334 ts_tablet_manager.cc:1428] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d: Time spent starting tablet: real 0.018s user 0.002s sys 0.006s
I20260806 03:41:38.217437 18334 tablet_bootstrap.cc:492] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d: Bootstrap starting.
I20260806 03:41:38.206436 18339 leader_election.cc:288] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:38.219956 18339 raft_consensus.cc:2781] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:41:38.220317 18339 raft_consensus.cc:482] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:41:38.220656 18339 raft_consensus.cc:3037] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:38.217973 18333 tablet_bootstrap.cc:492] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75: Bootstrap starting.
I20260806 03:41:38.225975 18334 tablet_bootstrap.cc:654] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:38.228413 18196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d" is_pre_election: true
W20260806 03:41:38.230155 18239 leader_election.cc:341] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729): Illegal state: must be running to vote when last-logged opid is not known
I20260806 03:41:38.245355 18332 ts_tablet_manager.cc:1428] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d: Time spent starting tablet: real 0.124s user 0.005s sys 0.007s
I20260806 03:41:38.246757 18332 tablet_bootstrap.cc:492] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d: Bootstrap starting.
I20260806 03:41:38.255331 18341 raft_consensus.cc:482] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:41:38.239540 18339 raft_consensus.cc:504] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:38.255906 18341 raft_consensus.cc:504] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:38.258867 18341 leader_election.cc:288] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:38.260217 18339 leader_election.cc:288] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 1 election: Requested vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:38.260401 18276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
I20260806 03:41:38.263034 18276 raft_consensus.cc:2454] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 491cdd4ce26949848b317caa81d5279d in term 0.
I20260806 03:41:38.266059 18159 leader_election.cc:302] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d, 6eb7ccdcc221406296e866e74a37ed75; no voters:
I20260806 03:41:38.265247 18334 tablet_bootstrap.cc:492] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d: No bootstrap required, opened a new log
I20260806 03:41:38.272651 18196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d" is_pre_election: true
I20260806 03:41:38.276615 18334 ts_tablet_manager.cc:1397] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d: Time spent bootstrapping tablet: real 0.059s user 0.021s sys 0.005s
I20260806 03:41:38.276507 18341 raft_consensus.cc:2781] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260806 03:41:38.279116 18078 leader_election.cc:341] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729): Illegal state: must be running to vote when last-logged opid is not known
I20260806 03:41:38.278268 18341 raft_consensus.cc:482] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:41:38.280380 18341 raft_consensus.cc:3037] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:38.282161 18332 tablet_bootstrap.cc:654] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:38.273404 18115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:38.283954 18276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
I20260806 03:41:38.284518 18113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d"
I20260806 03:41:38.284042 18115 raft_consensus.cc:2454] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 491cdd4ce26949848b317caa81d5279d in term 0.
I20260806 03:41:38.284926 18276 raft_consensus.cc:2379] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4abe04524a7046018757fcdfd13b815d in current term 1: Already voted for candidate 6eb7ccdcc221406296e866e74a37ed75 in this term.
I20260806 03:41:38.285748 18113 raft_consensus.cc:3037] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:38.284992 18196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d"
W20260806 03:41:38.288765 18239 leader_election.cc:341] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729): Illegal state: must be running to vote when last-logged opid is not known
I20260806 03:41:38.290789 18341 raft_consensus.cc:504] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:38.291450 18333 tablet_bootstrap.cc:654] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:38.292532 18079 leader_election.cc:302] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d; no voters: 491cdd4ce26949848b317caa81d5279d, 6eb7ccdcc221406296e866e74a37ed75
I20260806 03:41:38.293269 18341 leader_election.cc:288] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 1 election: Requested vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:38.294387 18276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75"
I20260806 03:41:38.295167 18276 raft_consensus.cc:3037] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:38.297420 18114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:38.298239 18114 raft_consensus.cc:2454] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6eb7ccdcc221406296e866e74a37ed75 in term 0.
I20260806 03:41:38.298651 18334 raft_consensus.cc:348] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:38.299889 18334 raft_consensus.cc:374] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:38.299750 18241 leader_election.cc:302] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75; no voters: 491cdd4ce26949848b317caa81d5279d
I20260806 03:41:38.300419 18334 raft_consensus.cc:729] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abe04524a7046018757fcdfd13b815d, State: Initialized, Role: FOLLOWER
I20260806 03:41:38.301398 18334 consensus_queue.cc:260] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:38.301672 18115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d"
I20260806 03:41:38.302572 18115 raft_consensus.cc:3037] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:38.308151 18334 ts_tablet_manager.cc:1428] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d: Time spent starting tablet: real 0.031s user 0.007s sys 0.002s
I20260806 03:41:38.313009 18276 raft_consensus.cc:2454] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 491cdd4ce26949848b317caa81d5279d in term 1.
I20260806 03:41:38.313741 18113 raft_consensus.cc:2454] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6eb7ccdcc221406296e866e74a37ed75 in term 1.
I20260806 03:41:38.315577 18339 raft_consensus.cc:2741] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260806 03:41:38.316377 18159 leader_election.cc:302] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d, 6eb7ccdcc221406296e866e74a37ed75; no voters:
I20260806 03:41:38.318089 18340 raft_consensus.cc:2726] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 03:41:38.320024 18341 raft_consensus.cc:2781] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:38.318014 18241 leader_election.cc:302] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75; no voters: 491cdd4ce26949848b317caa81d5279d
I20260806 03:41:38.323063 18339 raft_consensus.cc:2781] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:38.326601 18339 raft_consensus.cc:686] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 1 LEADER]: Becoming Leader. State: Replica: 6eb7ccdcc221406296e866e74a37ed75, State: Running, Role: LEADER
I20260806 03:41:38.327808 18339 consensus_queue.cc:237] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:38.331465 18341 raft_consensus.cc:686] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 1 LEADER]: Becoming Leader. State: Replica: 491cdd4ce26949848b317caa81d5279d, State: Running, Role: LEADER
I20260806 03:41:38.343864 18333 tablet_bootstrap.cc:492] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75: No bootstrap required, opened a new log
I20260806 03:41:38.344422 18333 ts_tablet_manager.cc:1397] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75: Time spent bootstrapping tablet: real 0.127s user 0.018s sys 0.017s
I20260806 03:41:38.344969 18332 tablet_bootstrap.cc:492] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d: No bootstrap required, opened a new log
I20260806 03:41:38.345551 18332 ts_tablet_manager.cc:1397] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d: Time spent bootstrapping tablet: real 0.099s user 0.014s sys 0.004s
I20260806 03:41:38.347411 18333 raft_consensus.cc:348] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:38.348623 18333 raft_consensus.cc:374] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:38.348971 18333 raft_consensus.cc:729] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eb7ccdcc221406296e866e74a37ed75, State: Initialized, Role: FOLLOWER
I20260806 03:41:38.333016 18341 consensus_queue.cc:237] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:38.350172 18333 consensus_queue.cc:260] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:38.351467 18332 raft_consensus.cc:348] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:38.352532 18332 raft_consensus.cc:374] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:38.352931 18332 raft_consensus.cc:729] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 491cdd4ce26949848b317caa81d5279d, State: Initialized, Role: FOLLOWER
I20260806 03:41:38.353737 18333 ts_tablet_manager.cc:1428] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75: Time spent starting tablet: real 0.009s user 0.003s sys 0.006s
I20260806 03:41:38.354147 18332 consensus_queue.cc:260] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:38.357559 18115 raft_consensus.cc:2454] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 491cdd4ce26949848b317caa81d5279d in term 1.
I20260806 03:41:38.360316 18332 ts_tablet_manager.cc:1428] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d: Time spent starting tablet: real 0.014s user 0.005s sys 0.005s
I20260806 03:41:38.365305 17906 catalog_manager.cc:5495] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67). New cstate: current_term: 1 leader_uuid: "6eb7ccdcc221406296e866e74a37ed75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } health_report { overall_health: HEALTHY } } }
I20260806 03:41:38.368510 18340 raft_consensus.cc:482] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:41:38.369112 18340 raft_consensus.cc:504] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:38.371486 18340 leader_election.cc:288] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363), 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:38.372856 18196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d" is_pre_election: true
I20260806 03:41:38.373909 18196 raft_consensus.cc:2454] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4abe04524a7046018757fcdfd13b815d in term 0.
I20260806 03:41:38.370669 17908 catalog_manager.cc:5495] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d reported cstate change: term changed from 0 to 1, leader changed from <none> to 491cdd4ce26949848b317caa81d5279d (127.15.252.66). New cstate: current_term: 1 leader_uuid: "491cdd4ce26949848b317caa81d5279d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } health_report { overall_health: UNKNOWN } } }
I20260806 03:41:38.383023 18078 leader_election.cc:302] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d, 4abe04524a7046018757fcdfd13b815d; no voters:
I20260806 03:41:38.384083 18340 raft_consensus.cc:2781] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:41:38.384521 18340 raft_consensus.cc:482] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:41:38.384927 18340 raft_consensus.cc:3037] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:38.394580 18276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
I20260806 03:41:38.395061 18340 raft_consensus.cc:504] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:38.395707 18276 raft_consensus.cc:2454] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4abe04524a7046018757fcdfd13b815d in term 0.
I20260806 03:41:38.398995 18196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d"
I20260806 03:41:38.399487 18275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75"
I20260806 03:41:38.399889 18196 raft_consensus.cc:3037] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:38.400203 18275 raft_consensus.cc:3037] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:38.410425 18275 raft_consensus.cc:2454] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4abe04524a7046018757fcdfd13b815d in term 1.
I20260806 03:41:38.412133 18079 leader_election.cc:302] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75; no voters:
I20260806 03:41:38.412428 18196 raft_consensus.cc:2454] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4abe04524a7046018757fcdfd13b815d in term 1.
I20260806 03:41:38.413750 18344 raft_consensus.cc:2781] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:38.406766 18340 leader_election.cc:288] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 1 election: Requested vote from peers 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363), 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:38.430857 18339 consensus_queue.cc:1035] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 03:41:38.436196 18344 raft_consensus.cc:686] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 1 LEADER]: Becoming Leader. State: Replica: 4abe04524a7046018757fcdfd13b815d, State: Running, Role: LEADER
I20260806 03:41:38.437438 18344 consensus_queue.cc:237] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:38.441977 18341 consensus_queue.cc:1035] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 03:41:38.447449 18341 raft_consensus.cc:482] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:41:38.448027 18341 raft_consensus.cc:504] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:38.451345 18341 leader_election.cc:288] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:38.451647 18196 raft_consensus.cc:3037] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:38.452329 18113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:38.456362 18276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
I20260806 03:41:38.458724 18159 leader_election.cc:302] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d; no voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75
I20260806 03:41:38.466095 18341 raft_consensus.cc:2726] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 03:41:38.480512 18339 consensus_queue.cc:1035] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 03:41:38.478440 18341 consensus_queue.cc:1035] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260806 03:41:38.617264 17908 catalog_manager.cc:5495] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4abe04524a7046018757fcdfd13b815d (127.15.252.65). New cstate: current_term: 1 leader_uuid: "4abe04524a7046018757fcdfd13b815d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } }
W20260806 03:41:38.688795 18374 log.cc:924] Time spent T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d: Append to log took a long time: real 0.060s user 0.006s sys 0.021s
I20260806 03:41:38.725070 18341 raft_consensus.cc:482] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6eb7ccdcc221406296e866e74a37ed75)
I20260806 03:41:38.725821 18341 raft_consensus.cc:504] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:38.728585 18341 leader_election.cc:288] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:38.730089 18113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:38.734206 18274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
I20260806 03:41:38.746263 18159 leader_election.cc:302] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d; no voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75
I20260806 03:41:38.757541 18386 raft_consensus.cc:2726] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260806 03:41:39.657783 18313 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.15.252.67:35363, user_credentials={real_user=slave}} blocked reactor thread for 45233.7us
I20260806 03:41:39.753810 18113 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54"
dest_uuid: "4abe04524a7046018757fcdfd13b815d"
from {username='slave'} at 127.0.0.1:53736
I20260806 03:41:39.755020 18113 raft_consensus.cc:580] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 1 LEADER]: Received request to step down
I20260806 03:41:39.755638 18113 raft_consensus.cc:3032] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 1 LEADER]: Stepping down as leader of term 1
I20260806 03:41:39.756147 18113 raft_consensus.cc:729] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4abe04524a7046018757fcdfd13b815d, State: Running, Role: LEADER
I20260806 03:41:39.757397 18113 consensus_queue.cc:260] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:39.759462 18113 raft_consensus.cc:3037] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:41:39.924360 18339 raft_consensus.cc:482] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4abe04524a7046018757fcdfd13b815d)
I20260806 03:41:39.925107 18339 raft_consensus.cc:504] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:39.927095 18339 leader_election.cc:288] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729), 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231)
I20260806 03:41:39.928525 18195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d" is_pre_election: true
I20260806 03:41:39.928876 18113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:39.929590 18113 raft_consensus.cc:2454] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6eb7ccdcc221406296e866e74a37ed75 in term 2.
I20260806 03:41:39.931301 18241 leader_election.cc:302] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75; no voters: 491cdd4ce26949848b317caa81d5279d
I20260806 03:41:39.932081 18339 raft_consensus.cc:2781] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 03:41:39.932673 18339 raft_consensus.cc:482] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4abe04524a7046018757fcdfd13b815d)
I20260806 03:41:39.933063 18339 raft_consensus.cc:3037] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:41:39.945973 18386 raft_consensus.cc:482] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4abe04524a7046018757fcdfd13b815d)
I20260806 03:41:39.946723 18386 raft_consensus.cc:504] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:39.948742 18386 leader_election.cc:288] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363), 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231)
I20260806 03:41:39.949921 18274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
I20260806 03:41:39.949900 18113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:39.951306 18113 raft_consensus.cc:2454] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 491cdd4ce26949848b317caa81d5279d in term 2.
I20260806 03:41:39.952400 18160 leader_election.cc:302] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d, 4abe04524a7046018757fcdfd13b815d; no voters:
I20260806 03:41:39.953399 18386 raft_consensus.cc:2781] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 03:41:39.953869 18386 raft_consensus.cc:482] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4abe04524a7046018757fcdfd13b815d)
I20260806 03:41:39.954401 18386 raft_consensus.cc:3037] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:41:39.977205 18339 raft_consensus.cc:504] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:39.977344 18386 raft_consensus.cc:504] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:39.980537 18195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d"
I20260806 03:41:39.981087 18274 raft_consensus.cc:2379] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 491cdd4ce26949848b317caa81d5279d in current term 2: Already voted for candidate 6eb7ccdcc221406296e866e74a37ed75 in this term.
I20260806 03:41:39.981575 18113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d"
I20260806 03:41:39.980579 18339 leader_election.cc:288] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 2 election: Requested vote from peers 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729), 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231)
I20260806 03:41:39.991465 18195 raft_consensus.cc:2379] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6eb7ccdcc221406296e866e74a37ed75 in current term 2: Already voted for candidate 491cdd4ce26949848b317caa81d5279d in this term.
I20260806 03:41:39.993422 18115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d"
I20260806 03:41:39.995527 18386 leader_election.cc:288] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 2 election: Requested vote from peers 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363), 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231)
I20260806 03:41:39.995741 18274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75"
I20260806 03:41:39.996918 18274 raft_consensus.cc:2379] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 491cdd4ce26949848b317caa81d5279d in current term 2: Already voted for candidate 6eb7ccdcc221406296e866e74a37ed75 in this term.
I20260806 03:41:39.996753 18195 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad"
dest_uuid: "491cdd4ce26949848b317caa81d5279d"
from {username='slave'} at 127.0.0.1:53062
I20260806 03:41:39.997686 18195 raft_consensus.cc:580] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 1 LEADER]: Received request to step down
I20260806 03:41:39.998082 18195 raft_consensus.cc:3032] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 1 LEADER]: Stepping down as leader of term 1
I20260806 03:41:39.998466 18195 raft_consensus.cc:729] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 491cdd4ce26949848b317caa81d5279d, State: Running, Role: LEADER
I20260806 03:41:39.999390 18195 consensus_queue.cc:260] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:40.002753 18195 raft_consensus.cc:3037] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:41:40.002590 18159 leader_election.cc:302] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d; no voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75
I20260806 03:41:40.005390 18386 raft_consensus.cc:2726] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 03:41:40.083750 18113 raft_consensus.cc:2454] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6eb7ccdcc221406296e866e74a37ed75 in term 2.
I20260806 03:41:40.086506 18241 leader_election.cc:302] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75; no voters: 491cdd4ce26949848b317caa81d5279d
I20260806 03:41:40.087416 18339 raft_consensus.cc:2781] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 03:41:40.088007 18339 raft_consensus.cc:686] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 2 LEADER]: Becoming Leader. State: Replica: 6eb7ccdcc221406296e866e74a37ed75, State: Running, Role: LEADER
I20260806 03:41:40.089097 18339 consensus_queue.cc:237] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:40.102205 17908 catalog_manager.cc:5495] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 reported cstate change: term changed from 1 to 2, leader changed from 4abe04524a7046018757fcdfd13b815d (127.15.252.65) to 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67). New cstate: current_term: 2 leader_uuid: "6eb7ccdcc221406296e866e74a37ed75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } health_report { overall_health: UNKNOWN } } }
I20260806 03:41:40.170905 18195 raft_consensus.cc:1261] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Refusing update from remote peer 6eb7ccdcc221406296e866e74a37ed75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260806 03:41:40.172547 18339 consensus_queue.cc:1035] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260806 03:41:40.173283 18113 raft_consensus.cc:1261] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Refusing update from remote peer 6eb7ccdcc221406296e866e74a37ed75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260806 03:41:40.175189 18339 consensus_queue.cc:1035] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260806 03:41:40.190022 18339 raft_consensus.cc:482] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 491cdd4ce26949848b317caa81d5279d)
I20260806 03:41:40.190636 18339 raft_consensus.cc:504] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:40.197434 18339 leader_election.cc:288] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:40.201908 18113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:40.202988 18113 raft_consensus.cc:2454] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6eb7ccdcc221406296e866e74a37ed75 in term 1.
I20260806 03:41:40.216293 18241 leader_election.cc:302] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75; no voters:
I20260806 03:41:40.220952 18406 raft_consensus.cc:2781] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 03:41:40.221487 18406 raft_consensus.cc:482] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 491cdd4ce26949848b317caa81d5279d)
I20260806 03:41:40.221972 18406 raft_consensus.cc:3037] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:41:40.225782 18195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d" is_pre_election: true
I20260806 03:41:40.226531 18195 raft_consensus.cc:2454] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6eb7ccdcc221406296e866e74a37ed75 in term 2.
I20260806 03:41:40.259495 18274 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54"
dest_uuid: "6eb7ccdcc221406296e866e74a37ed75"
from {username='slave'} at 127.0.0.1:56408
I20260806 03:41:40.260331 18274 raft_consensus.cc:580] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 2 LEADER]: Received request to step down
I20260806 03:41:40.260715 18274 raft_consensus.cc:3032] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 2 LEADER]: Stepping down as leader of term 2
I20260806 03:41:40.260995 18274 raft_consensus.cc:729] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6eb7ccdcc221406296e866e74a37ed75, State: Running, Role: LEADER
I20260806 03:41:40.262297 18274 consensus_queue.cc:260] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:40.263481 18274 raft_consensus.cc:3037] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 2 FOLLOWER]: Advancing to term 3
I20260806 03:41:40.278065 18406 raft_consensus.cc:504] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:40.280313 18406 leader_election.cc:288] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 2 election: Requested vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:40.281327 18115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d"
I20260806 03:41:40.282130 18115 raft_consensus.cc:3037] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:41:40.282038 18196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d"
I20260806 03:41:40.382383 18115 raft_consensus.cc:2454] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6eb7ccdcc221406296e866e74a37ed75 in term 2.
I20260806 03:41:40.382325 18196 raft_consensus.cc:2454] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6eb7ccdcc221406296e866e74a37ed75 in term 2.
I20260806 03:41:40.383838 18241 leader_election.cc:302] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75; no voters:
I20260806 03:41:40.384800 18406 raft_consensus.cc:2781] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 03:41:40.385370 18406 raft_consensus.cc:686] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 2 LEADER]: Becoming Leader. State: Replica: 6eb7ccdcc221406296e866e74a37ed75, State: Running, Role: LEADER
I20260806 03:41:40.386289 18406 consensus_queue.cc:237] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:40.397444 17906 catalog_manager.cc:5495] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 reported cstate change: term changed from 1 to 2, leader changed from 491cdd4ce26949848b317caa81d5279d (127.15.252.66) to 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67). New cstate: current_term: 2 leader_uuid: "6eb7ccdcc221406296e866e74a37ed75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } health_report { overall_health: HEALTHY } } }
I20260806 03:41:40.430382 18386 raft_consensus.cc:482] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6eb7ccdcc221406296e866e74a37ed75)
I20260806 03:41:40.430994 18386 raft_consensus.cc:504] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:40.433598 18386 leader_election.cc:288] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363), 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231)
I20260806 03:41:40.433977 18274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
I20260806 03:41:40.434422 18115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:40.434850 18274 raft_consensus.cc:2454] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 491cdd4ce26949848b317caa81d5279d in term 3.
I20260806 03:41:40.436044 18159 leader_election.cc:302] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d, 6eb7ccdcc221406296e866e74a37ed75; no voters: 4abe04524a7046018757fcdfd13b815d
I20260806 03:41:40.436880 18386 raft_consensus.cc:2781] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 03:41:40.437218 18386 raft_consensus.cc:482] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6eb7ccdcc221406296e866e74a37ed75)
I20260806 03:41:40.437598 18386 raft_consensus.cc:3037] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Advancing to term 3
I20260806 03:41:40.461750 18340 raft_consensus.cc:482] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6eb7ccdcc221406296e866e74a37ed75)
I20260806 03:41:40.462363 18340 raft_consensus.cc:504] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:40.464461 18340 leader_election.cc:288] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363), 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:40.465330 18274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
I20260806 03:41:40.466020 18274 raft_consensus.cc:2454] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4abe04524a7046018757fcdfd13b815d in term 3.
I20260806 03:41:40.473322 18079 leader_election.cc:302] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75; no voters:
I20260806 03:41:40.475133 18340 raft_consensus.cc:2781] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 03:41:40.474424 18274 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821"
dest_uuid: "6eb7ccdcc221406296e866e74a37ed75"
from {username='slave'} at 127.0.0.1:56408
I20260806 03:41:40.475536 18340 raft_consensus.cc:482] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6eb7ccdcc221406296e866e74a37ed75)
I20260806 03:41:40.475800 18274 raft_consensus.cc:580] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 1 LEADER]: Received request to step down
I20260806 03:41:40.475983 18340 raft_consensus.cc:3037] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Advancing to term 3
I20260806 03:41:40.476244 18274 raft_consensus.cc:3032] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 1 LEADER]: Stepping down as leader of term 1
I20260806 03:41:40.476613 18274 raft_consensus.cc:729] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6eb7ccdcc221406296e866e74a37ed75, State: Running, Role: LEADER
I20260806 03:41:40.477402 18274 consensus_queue.cc:260] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:40.478783 18274 raft_consensus.cc:3037] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:41:40.471915 18196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d" is_pre_election: true
I20260806 03:41:40.482306 18195 raft_consensus.cc:1261] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Refusing update from remote peer 6eb7ccdcc221406296e866e74a37ed75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260806 03:41:40.483753 18406 consensus_queue.cc:1035] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260806 03:41:40.486696 18386 raft_consensus.cc:504] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:40.489019 18196 raft_consensus.cc:2379] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4abe04524a7046018757fcdfd13b815d in current term 3: Already voted for candidate 491cdd4ce26949848b317caa81d5279d in this term.
I20260806 03:41:40.486764 18340 raft_consensus.cc:504] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:40.491446 18340 leader_election.cc:288] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 3 election: Requested vote from peers 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363), 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:40.492837 18386 leader_election.cc:288] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 3 election: Requested vote from peers 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363), 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231)
I20260806 03:41:40.494490 18274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75"
I20260806 03:41:40.495630 18276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75"
I20260806 03:41:40.498147 18115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d"
I20260806 03:41:40.498963 18115 raft_consensus.cc:2379] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 491cdd4ce26949848b317caa81d5279d in current term 3: Already voted for candidate 4abe04524a7046018757fcdfd13b815d in this term.
I20260806 03:41:40.502216 18195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d"
I20260806 03:41:40.504262 18078 leader_election.cc:302] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d; no voters: 491cdd4ce26949848b317caa81d5279d, 6eb7ccdcc221406296e866e74a37ed75
I20260806 03:41:40.504698 18115 raft_consensus.cc:1261] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Refusing update from remote peer 6eb7ccdcc221406296e866e74a37ed75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260806 03:41:40.507197 18406 consensus_queue.cc:1035] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260806 03:41:40.513726 18340 raft_consensus.cc:2726] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260806 03:41:40.565696 18340 raft_consensus.cc:482] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6eb7ccdcc221406296e866e74a37ed75)
I20260806 03:41:40.566298 18340 raft_consensus.cc:504] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:40.567994 18340 leader_election.cc:288] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:40.569119 18196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d" is_pre_election: true
I20260806 03:41:40.569314 18276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
I20260806 03:41:40.570202 18276 raft_consensus.cc:2454] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4abe04524a7046018757fcdfd13b815d in term 2.
I20260806 03:41:40.571623 18079 leader_election.cc:302] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75; no voters: 491cdd4ce26949848b317caa81d5279d
I20260806 03:41:40.572479 18340 raft_consensus.cc:2781] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 03:41:40.572918 18340 raft_consensus.cc:482] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6eb7ccdcc221406296e866e74a37ed75)
I20260806 03:41:40.573722 18340 raft_consensus.cc:3037] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:41:40.591928 18274 raft_consensus.cc:2454] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 491cdd4ce26949848b317caa81d5279d in term 3.
I20260806 03:41:40.591962 18340 raft_consensus.cc:504] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:40.593602 18159 leader_election.cc:302] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d, 6eb7ccdcc221406296e866e74a37ed75; no voters: 4abe04524a7046018757fcdfd13b815d
I20260806 03:41:40.594596 18386 raft_consensus.cc:2781] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Leader election won for term 3
I20260806 03:41:40.595157 18386 raft_consensus.cc:686] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 3 LEADER]: Becoming Leader. State: Replica: 491cdd4ce26949848b317caa81d5279d, State: Running, Role: LEADER
I20260806 03:41:40.596436 18340 leader_election.cc:288] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 2 election: Requested vote from peers 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:40.596717 18386 consensus_queue.cc:237] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:40.597128 18274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75"
I20260806 03:41:40.599875 18196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d"
I20260806 03:41:40.606614 18386 raft_consensus.cc:482] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6eb7ccdcc221406296e866e74a37ed75)
I20260806 03:41:40.606563 17908 catalog_manager.cc:5495] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d reported cstate change: term changed from 2 to 3, leader changed from 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67) to 491cdd4ce26949848b317caa81d5279d (127.15.252.66). New cstate: current_term: 3 leader_uuid: "491cdd4ce26949848b317caa81d5279d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } health_report { overall_health: UNKNOWN } } }
I20260806 03:41:40.607651 18386 raft_consensus.cc:504] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:40.610795 18115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:40.611052 18274 raft_consensus.cc:2454] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4abe04524a7046018757fcdfd13b815d in term 2.
I20260806 03:41:40.611685 18115 raft_consensus.cc:2379] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 491cdd4ce26949848b317caa81d5279d in current term 2: Already voted for candidate 4abe04524a7046018757fcdfd13b815d in this term.
I20260806 03:41:40.612419 18386 leader_election.cc:288] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:40.613423 18274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
I20260806 03:41:40.614290 18274 raft_consensus.cc:2379] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 491cdd4ce26949848b317caa81d5279d in current term 2: Already voted for candidate 4abe04524a7046018757fcdfd13b815d in this term.
I20260806 03:41:40.615869 18159 leader_election.cc:302] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d; no voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75
I20260806 03:41:40.616106 18079 leader_election.cc:302] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75; no voters: 491cdd4ce26949848b317caa81d5279d
I20260806 03:41:40.617136 18386 raft_consensus.cc:3037] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:41:40.618423 18340 raft_consensus.cc:2781] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Leader election won for term 2
I20260806 03:41:40.624544 18386 raft_consensus.cc:2726] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:41:40.634481 18340 raft_consensus.cc:686] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 2 LEADER]: Becoming Leader. State: Replica: 4abe04524a7046018757fcdfd13b815d, State: Running, Role: LEADER
I20260806 03:41:40.635586 18340 consensus_queue.cc:237] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:40.649287 17906 catalog_manager.cc:5495] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d reported cstate change: term changed from 1 to 2, leader changed from 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67) to 4abe04524a7046018757fcdfd13b815d (127.15.252.65). New cstate: current_term: 2 leader_uuid: "4abe04524a7046018757fcdfd13b815d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } health_report { overall_health: UNKNOWN } } }
I20260806 03:41:40.678288 18115 raft_consensus.cc:1261] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Refusing update from remote peer 491cdd4ce26949848b317caa81d5279d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260806 03:41:40.679728 18386 consensus_queue.cc:1035] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260806 03:41:40.683877 18274 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad"
dest_uuid: "6eb7ccdcc221406296e866e74a37ed75"
from {username='slave'} at 127.0.0.1:56408
I20260806 03:41:40.684499 18274 raft_consensus.cc:580] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 2 LEADER]: Received request to step down
I20260806 03:41:40.684859 18274 raft_consensus.cc:3032] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 2 LEADER]: Stepping down as leader of term 2
I20260806 03:41:40.685199 18274 raft_consensus.cc:729] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6eb7ccdcc221406296e866e74a37ed75, State: Running, Role: LEADER
I20260806 03:41:40.685958 18274 consensus_queue.cc:260] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:40.687726 18274 raft_consensus.cc:3037] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 2 FOLLOWER]: Advancing to term 3
I20260806 03:41:40.699008 18274 raft_consensus.cc:1261] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Refusing update from remote peer 491cdd4ce26949848b317caa81d5279d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260806 03:41:40.701334 18440 consensus_queue.cc:1035] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260806 03:41:40.723512 18196 raft_consensus.cc:1261] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Refusing update from remote peer 4abe04524a7046018757fcdfd13b815d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260806 03:41:40.725587 18340 consensus_queue.cc:1035] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [LEADER]: Connected to new peer: Peer: permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260806 03:41:40.742741 18274 raft_consensus.cc:1261] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 2 FOLLOWER]: Refusing update from remote peer 4abe04524a7046018757fcdfd13b815d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260806 03:41:40.744603 18439 consensus_queue.cc:1035] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260806 03:41:40.801034 18440 raft_consensus.cc:482] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6eb7ccdcc221406296e866e74a37ed75)
I20260806 03:41:40.801561 18440 raft_consensus.cc:504] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:40.803233 18440 leader_election.cc:288] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:40.804148 18115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:40.804296 18274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
I20260806 03:41:40.805049 18274 raft_consensus.cc:2454] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 491cdd4ce26949848b317caa81d5279d in term 3.
I20260806 03:41:40.806178 18159 leader_election.cc:302] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d, 6eb7ccdcc221406296e866e74a37ed75; no voters: 4abe04524a7046018757fcdfd13b815d
I20260806 03:41:40.807039 18440 raft_consensus.cc:2781] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 03:41:40.807477 18440 raft_consensus.cc:482] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6eb7ccdcc221406296e866e74a37ed75)
I20260806 03:41:40.807817 18440 raft_consensus.cc:3037] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Advancing to term 3
I20260806 03:41:40.812672 18440 raft_consensus.cc:504] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:40.814373 18440 leader_election.cc:288] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 3 election: Requested vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:40.815330 18115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d"
I20260806 03:41:40.815531 18274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75"
I20260806 03:41:40.821290 18274 raft_consensus.cc:2454] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 491cdd4ce26949848b317caa81d5279d in term 3.
I20260806 03:41:40.822430 18159 leader_election.cc:302] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d, 6eb7ccdcc221406296e866e74a37ed75; no voters: 4abe04524a7046018757fcdfd13b815d
I20260806 03:41:40.823280 18440 raft_consensus.cc:2781] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Leader election won for term 3
I20260806 03:41:40.823799 18440 raft_consensus.cc:686] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 3 LEADER]: Becoming Leader. State: Replica: 491cdd4ce26949848b317caa81d5279d, State: Running, Role: LEADER
I20260806 03:41:40.824678 18440 consensus_queue.cc:237] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:40.830598 18340 raft_consensus.cc:482] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6eb7ccdcc221406296e866e74a37ed75)
I20260806 03:41:40.831645 18340 raft_consensus.cc:504] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:40.833393 18340 leader_election.cc:288] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:40.833300 17905 catalog_manager.cc:5495] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d reported cstate change: term changed from 2 to 3, leader changed from 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67) to 491cdd4ce26949848b317caa81d5279d (127.15.252.66). New cstate: current_term: 3 leader_uuid: "491cdd4ce26949848b317caa81d5279d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } health_report { overall_health: UNKNOWN } } }
I20260806 03:41:40.834179 18196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d" is_pre_election: true
I20260806 03:41:40.834414 18274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
I20260806 03:41:40.835208 18274 raft_consensus.cc:2379] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4abe04524a7046018757fcdfd13b815d in current term 3: Already voted for candidate 491cdd4ce26949848b317caa81d5279d in this term.
I20260806 03:41:40.836189 18079 leader_election.cc:302] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d; no voters: 491cdd4ce26949848b317caa81d5279d, 6eb7ccdcc221406296e866e74a37ed75
I20260806 03:41:40.836850 18340 raft_consensus.cc:3037] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Advancing to term 3
I20260806 03:41:40.845322 18340 raft_consensus.cc:2726] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260806 03:41:40.896142 17908 catalog_manager.cc:3333] Servicing AlterTable request from {username='slave'} at 127.0.0.1:53732:
table { table_id: "325a9a6c58fd4d258c4388b45595aefe" table_name: "test-table" } new_table_name: "NewTableName"
I20260806 03:41:40.909852 18274 raft_consensus.cc:1261] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Refusing update from remote peer 491cdd4ce26949848b317caa81d5279d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260806 03:41:40.910890 18113 raft_consensus.cc:1261] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Refusing update from remote peer 491cdd4ce26949848b317caa81d5279d: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260806 03:41:40.911705 18386 consensus_queue.cc:1035] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260806 03:41:40.912632 18440 consensus_queue.cc:1035] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260806 03:41:40.951558 16369 master.cc:537] Master@127.15.252.125:34597 shutting down...
I20260806 03:41:40.989605 17906 catalog_manager.cc:5587] TS 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231) does not have the latest schema for tablet 6c23a83f61a74931bb2a5239d32e1cad (table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe]). Expected version 1 got 0
I20260806 03:41:41.024068 17905 catalog_manager.cc:5587] TS 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363) does not have the latest schema for tablet 6c23a83f61a74931bb2a5239d32e1cad (table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe]). Expected version 1 got 0
I20260806 03:41:41.033231 18448 tablet.cc:1722] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 03:41:41.043915 18450 tablet.cc:1722] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 03:41:41.071655 18451 tablet.cc:1722] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 03:41:41.075250 18453 tablet.cc:1722] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 03:41:41.088394 18452 tablet.cc:1722] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 03:41:41.118880 18458 tablet.cc:1722] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 03:41:41.126210 18454 tablet.cc:1722] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 03:41:41.130977 18449 tablet.cc:1722] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 03:41:41.128855 18457 tablet.cc:1722] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 03:41:41.242547 16369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:41:41.243953 16369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:41.244616 16369 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561: stopping tablet replica
I20260806 03:41:41.269865 16369 master.cc:552] Master@127.15.252.125:34597 shutdown complete.
I20260806 03:41:41.336740 18460 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2a5ff5ca39d348fa837104d428536561)
I20260806 03:41:41.337220 18460 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:41.339443 18460 leader_election.cc:288] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 30063023a8844495837679314257b563 (127.15.252.126:42845), 2a5ff5ca39d348fa837104d428536561 (127.15.252.125:34597)
I20260806 03:41:41.341173 17852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" candidate_term: 2 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: false dest_uuid: "30063023a8844495837679314257b563" is_pre_election: true
W20260806 03:41:41.346714 17960 leader_election.cc:334] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2a5ff5ca39d348fa837104d428536561 (127.15.252.125:34597): Network error: Client connection negotiation failed: client connection to 127.15.252.125:34597: connect: Connection refused (error 111)
I20260806 03:41:41.347400 17960 leader_election.cc:302] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c0591ed664d442fdaebaa2cd64c84b9f; no voters: 2a5ff5ca39d348fa837104d428536561, 30063023a8844495837679314257b563
I20260806 03:41:41.348340 18460 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:41:41.435459 18465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2a5ff5ca39d348fa837104d428536561)
I20260806 03:41:41.436039 18465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:41.438015 18465 leader_election.cc:288] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2a5ff5ca39d348fa837104d428536561 (127.15.252.125:34597), c0591ed664d442fdaebaa2cd64c84b9f (127.15.252.124:39623)
I20260806 03:41:41.439354 17984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30063023a8844495837679314257b563" candidate_term: 2 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: false dest_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" is_pre_election: true
I20260806 03:41:41.439970 17984 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30063023a8844495837679314257b563 in term 1.
I20260806 03:41:41.441143 17828 leader_election.cc:302] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30063023a8844495837679314257b563, c0591ed664d442fdaebaa2cd64c84b9f; no voters:
I20260806 03:41:41.442010 18465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 03:41:41.442445 18465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2a5ff5ca39d348fa837104d428536561)
I20260806 03:41:41.442855 18465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 1 FOLLOWER]: Advancing to term 2
W20260806 03:41:41.444223 17829 leader_election.cc:334] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2a5ff5ca39d348fa837104d428536561 (127.15.252.125:34597): Network error: Client connection negotiation failed: client connection to 127.15.252.125:34597: connect: Connection refused (error 111)
I20260806 03:41:41.449378 18465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:41.451347 18465 leader_election.cc:288] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [CANDIDATE]: Term 2 election: Requested vote from peers 2a5ff5ca39d348fa837104d428536561 (127.15.252.125:34597), c0591ed664d442fdaebaa2cd64c84b9f (127.15.252.124:39623)
I20260806 03:41:41.453276 17984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30063023a8844495837679314257b563" candidate_term: 2 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: false dest_uuid: "c0591ed664d442fdaebaa2cd64c84b9f"
I20260806 03:41:41.453966 17984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:41:41.461427 17984 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30063023a8844495837679314257b563 in term 2.
I20260806 03:41:41.462433 17828 leader_election.cc:302] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30063023a8844495837679314257b563, c0591ed664d442fdaebaa2cd64c84b9f; no voters:
I20260806 03:41:41.463447 18465 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 03:41:41.464404 18465 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 2 LEADER]: Becoming Leader. State: Replica: 30063023a8844495837679314257b563, State: Running, Role: LEADER
I20260806 03:41:41.465302 18465 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 1.15, Last appended by leader: 15, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:41.469695 18468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30063023a8844495837679314257b563. Latest consensus state: current_term: 2 leader_uuid: "30063023a8844495837679314257b563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:41.470402 18468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:41.471958 18469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:41:41.479058 18469 catalog_manager.cc:669] Loaded metadata for table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe]
I20260806 03:41:41.485333 18469 tablet_loader.cc:96] loaded metadata for tablet 4f6ce5ed631f413d907292ee85373f54 (table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe])
I20260806 03:41:41.487262 18469 tablet_loader.cc:96] loaded metadata for tablet 5390b9a07b5f4cd8b2330fd0974c8821 (table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe])
I20260806 03:41:41.488758 18469 tablet_loader.cc:96] loaded metadata for tablet 6c23a83f61a74931bb2a5239d32e1cad (table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe])
I20260806 03:41:41.490212 18469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:41:41.492332 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:41:41.495958 18469 catalog_manager.cc:1259] Loaded cluster ID: 8225191a001148569992571dec22b329
I20260806 03:41:41.496359 18469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
W20260806 03:41:41.501183 18471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:41.501179 18470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:41.505797 18469 catalog_manager.cc:1504] Loading token signing keys...
W20260806 03:41:41.506573 18474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:41.506829 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:41.508033 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:41.508373 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:41.508612 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987701508600 us; error 0 us; skew 500 ppm
I20260806 03:41:41.509228 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:41.510288 18469 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563: Loaded TSK: 0
I20260806 03:41:41.511848 18469 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:41:41.512964 16369 webserver.cc:460] Webserver started at http://127.15.252.125:37953/ using document root <none> and password file <none>
I20260806 03:41:41.513609 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:41.513926 16369 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260806 03:41:41.527666 16369 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.003s sys 0.000s
I20260806 03:41:41.531476 18479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:41.532392 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 03:41:41.532757 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-1-root
uuid: "2a5ff5ca39d348fa837104d428536561"
format_stamp: "Formatted at 2026-08-06 03:41:36 on dist-test-slave-csg5"
I20260806 03:41:41.533134 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:41.557780 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:41.559685 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:41.575764 17984 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 2 FOLLOWER]: Refusing update from remote peer 30063023a8844495837679314257b563: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 2 index: 16. (index mismatch)
I20260806 03:41:41.577663 18468 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260806 03:41:41.582113 18460 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30063023a8844495837679314257b563. Latest consensus state: current_term: 2 leader_uuid: "30063023a8844495837679314257b563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:41.583544 18460 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:41.586484 18465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "30063023a8844495837679314257b563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:41.587373 18465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:41.590402 18460 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "30063023a8844495837679314257b563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:41.591320 18460 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:41.614337 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.125:34597
I20260806 03:41:41.614456 18533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.125:34597 every 8 connection(s)
I20260806 03:41:41.620813 18142 heartbeater.cc:503] Master 127.15.252.126:42845 was elected leader, sending a full tablet report...
I20260806 03:41:41.630285 18535 sys_catalog.cc:263] Verifying existing consensus state
I20260806 03:41:41.636982 18141 heartbeater.cc:347] Connected to a master server at 127.15.252.125:34597
I20260806 03:41:41.637743 18535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561: Bootstrap starting.
I20260806 03:41:41.649545 17842 catalog_manager.cc:6040] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563: NewTableName [id=325a9a6c58fd4d258c4388b45595aefe] alter complete (version 1)
I20260806 03:41:41.664692 18224 heartbeater.cc:503] Master 127.15.252.126:42845 was elected leader, sending a full tablet report...
I20260806 03:41:41.666015 18305 heartbeater.cc:503] Master 127.15.252.126:42845 was elected leader, sending a full tablet report...
I20260806 03:41:41.679454 18304 heartbeater.cc:347] Connected to a master server at 127.15.252.125:34597
W20260806 03:41:41.699641 17829 leader_election.cc:334] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2a5ff5ca39d348fa837104d428536561 (127.15.252.125:34597): Timed out: connection negotiation to 127.15.252.125:34597 for RPC RequestConsensusVote timed out after 0.245s (ON_OUTBOUND_QUEUE)
I20260806 03:41:41.701144 18223 heartbeater.cc:347] Connected to a master server at 127.15.252.125:34597
I20260806 03:41:41.711980 18535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=0 applied=15 ignored=0} inserts{seen=7 ignored=0} mutations{seen=12 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:41:41.713507 18535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561: Bootstrap complete.
I20260806 03:41:41.716459 18535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:41.717023 18535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a5ff5ca39d348fa837104d428536561, State: Initialized, Role: FOLLOWER
I20260806 03:41:41.717633 18535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 15, Last appended: 1.15, Last appended by leader: 15, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:41.719501 18542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:41.720300 18542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:41.720713 18535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:41:41.726411 16369 master.cc:537] Master@127.15.252.126:42845 shutting down...
I20260806 03:41:41.784716 18553 catalog_manager.cc:1259] Loaded cluster ID: 8225191a001148569992571dec22b329
I20260806 03:41:41.785054 18553 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561: loading cluster ID for follower catalog manager: success
I20260806 03:41:41.790261 16369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 2 LEADER]: Raft consensus shutting down.
I20260806 03:41:41.791252 16369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:41.791746 16369 tablet_replica.cc:330] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563: stopping tablet replica
I20260806 03:41:41.792693 18553 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561: acquiring CA information for follower catalog manager: success
I20260806 03:41:41.797936 18553 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 03:41:41.861375 18497 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "4abe04524a7046018757fcdfd13b815d" instance_seqno: 1785987697088220) as {username='slave'} at 127.0.0.1:53884; Asking this server to re-register.
I20260806 03:41:41.861482 18494 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "491cdd4ce26949848b317caa81d5279d" instance_seqno: 1785987697259302) as {username='slave'} at 127.0.0.1:53910; Asking this server to re-register.
I20260806 03:41:41.861989 18496 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" instance_seqno: 1785987697497837) as {username='slave'} at 127.0.0.1:53900; Asking this server to re-register.
I20260806 03:41:41.863538 18223 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:41.863691 18141 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:41.864403 18304 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:41.867597 18494 ts_manager.cc:194] Registered new tserver with Master: 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:41.867954 18496 ts_manager.cc:194] Registered new tserver with Master: 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231)
I20260806 03:41:41.868417 18497 ts_manager.cc:194] Registered new tserver with Master: 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:41.953605 18460 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 30063023a8844495837679314257b563)
I20260806 03:41:41.954226 18460 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:41.955991 18460 leader_election.cc:288] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 30063023a8844495837679314257b563 (127.15.252.126:42845), 2a5ff5ca39d348fa837104d428536561 (127.15.252.125:34597)
W20260806 03:41:41.957202 17958 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master [suppressed 10 similar messages]
I20260806 03:41:41.959786 18542 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:41:41.960327 18542 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
W20260806 03:41:41.962695 17958 leader_election.cc:334] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 30063023a8844495837679314257b563 (127.15.252.126:42845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260806 03:41:41.963670 18542 leader_election.cc:288] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 30063023a8844495837679314257b563 (127.15.252.126:42845), c0591ed664d442fdaebaa2cd64c84b9f (127.15.252.124:39623)
I20260806 03:41:41.979655 18509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" candidate_term: 3 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: false dest_uuid: "2a5ff5ca39d348fa837104d428536561" is_pre_election: true
I20260806 03:41:41.980656 18509 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0591ed664d442fdaebaa2cd64c84b9f in term 1.
I20260806 03:41:41.981914 17960 leader_election.cc:302] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a5ff5ca39d348fa837104d428536561, c0591ed664d442fdaebaa2cd64c84b9f; no voters: 30063023a8844495837679314257b563
I20260806 03:41:41.982812 18460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 03:41:41.983176 18460 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 30063023a8844495837679314257b563)
I20260806 03:41:41.983496 18460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 2 FOLLOWER]: Advancing to term 3
I20260806 03:41:41.991856 18460 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
W20260806 03:41:41.992717 18481 leader_election.cc:334] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 30063023a8844495837679314257b563 (127.15.252.126:42845): Network error: Client connection negotiation failed: client connection to 127.15.252.126:42845: connect: Connection refused (error 111)
I20260806 03:41:41.995710 17984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a5ff5ca39d348fa837104d428536561" candidate_term: 2 candidate_status { last_received { term: 1 index: 15 } } ignore_live_leader: false dest_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" is_pre_election: true
I20260806 03:41:41.995568 18460 leader_election.cc:288] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [CANDIDATE]: Term 3 election: Requested vote from peers 30063023a8844495837679314257b563 (127.15.252.126:42845), 2a5ff5ca39d348fa837104d428536561 (127.15.252.125:34597)
I20260806 03:41:41.996598 17984 raft_consensus.cc:2354] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2a5ff5ca39d348fa837104d428536561 for earlier term 2. Current term is 3.
I20260806 03:41:41.996591 18509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" candidate_term: 3 candidate_status { last_received { term: 2 index: 17 } } ignore_live_leader: false dest_uuid: "2a5ff5ca39d348fa837104d428536561"
I20260806 03:41:41.997300 18509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 1 FOLLOWER]: Advancing to term 3
W20260806 03:41:41.997058 17958 leader_election.cc:334] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 30063023a8844495837679314257b563 (127.15.252.126:42845): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260806 03:41:41.997937 18482 leader_election.cc:398] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [CANDIDATE]: Term 2 pre-election: Vote denied by peer c0591ed664d442fdaebaa2cd64c84b9f (127.15.252.124:39623) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2a5ff5ca39d348fa837104d428536561 for earlier term 2. Current term is 3.
I20260806 03:41:41.998296 18482 leader_election.cc:401] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260806 03:41:42.002558 18509 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0591ed664d442fdaebaa2cd64c84b9f in term 3.
I20260806 03:41:42.003091 18542 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer c0591ed664d442fdaebaa2cd64c84b9f (127.15.252.124:39623) with higher term. Message: Invalid argument: T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2a5ff5ca39d348fa837104d428536561 for earlier term 2. Current term is 3.
I20260806 03:41:42.003924 17960 leader_election.cc:302] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a5ff5ca39d348fa837104d428536561, c0591ed664d442fdaebaa2cd64c84b9f; no voters: 30063023a8844495837679314257b563
I20260806 03:41:42.004798 18460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 3 FOLLOWER]: Leader election won for term 3
I20260806 03:41:42.006044 18460 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 3 LEADER]: Becoming Leader. State: Replica: c0591ed664d442fdaebaa2cd64c84b9f, State: Running, Role: LEADER
I20260806 03:41:42.007025 18460 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 2.17, Last appended by leader: 17, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:42.011001 18560 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0591ed664d442fdaebaa2cd64c84b9f. Latest consensus state: current_term: 3 leader_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:42.011605 18560 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:42.013355 18561 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:41:42.018966 18561 catalog_manager.cc:669] Loaded metadata for table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe]
I20260806 03:41:42.023589 18561 tablet_loader.cc:96] loaded metadata for tablet 4f6ce5ed631f413d907292ee85373f54 (table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe])
I20260806 03:41:42.024704 18561 tablet_loader.cc:96] loaded metadata for tablet 5390b9a07b5f4cd8b2330fd0974c8821 (table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe])
I20260806 03:41:42.025842 18561 tablet_loader.cc:96] loaded metadata for tablet 6c23a83f61a74931bb2a5239d32e1cad (table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe])
I20260806 03:41:42.026880 18561 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:41:42.030292 18561 catalog_manager.cc:1259] Loaded cluster ID: 8225191a001148569992571dec22b329
I20260806 03:41:42.030640 18561 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:41:42.035197 18561 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:41:42.038805 18561 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f: Loaded TSK: 0
I20260806 03:41:42.040251 18561 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 03:41:42.096309 17958 consensus_peers.cc:487] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f -> Peer 30063023a8844495837679314257b563 (127.15.252.126:42845): Couldn't send request to peer 30063023a8844495837679314257b563. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 03:41:42.124625 18509 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 3 FOLLOWER]: Refusing update from remote peer c0591ed664d442fdaebaa2cd64c84b9f: Log matching property violated. Preceding OpId in replica: term: 1 index: 15. Preceding OpId from leader: term: 3 index: 18. (index mismatch)
I20260806 03:41:42.126044 18560 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.000s
I20260806 03:41:42.136765 18564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:42.137076 18542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0591ed664d442fdaebaa2cd64c84b9f. Latest consensus state: current_term: 3 leader_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:42.137692 18564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:42.137854 18542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:42.138139 18460 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:42.138968 18460 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:42.139911 18542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:42.140897 18542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:42.172382 18303 heartbeater.cc:503] Master 127.15.252.124:39623 was elected leader, sending a full tablet report...
I20260806 03:41:42.181071 18140 heartbeater.cc:503] Master 127.15.252.124:39623 was elected leader, sending a full tablet report...
I20260806 03:41:42.182961 18222 heartbeater.cc:503] Master 127.15.252.124:39623 was elected leader, sending a full tablet report...
I20260806 03:41:42.491569 18509 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 3 FOLLOWER]: Rejecting Update request from peer 30063023a8844495837679314257b563 for earlier term 2. Current term is 3. Ops: []
I20260806 03:41:42.511826 16369 master.cc:552] Master@127.15.252.126:42845 shutdown complete.
W20260806 03:41:42.584486 17958 consensus_peers.cc:487] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f -> Peer 30063023a8844495837679314257b563 (127.15.252.126:42845): Couldn't send request to peer 30063023a8844495837679314257b563. Status: Network error: Client connection negotiation failed: client connection to 127.15.252.126:42845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260806 03:41:42.731086 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:42.737306 18571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:42.739583 18572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:42.740814 18574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:42.741533 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:42.743043 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:42.743270 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:42.743412 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987702743402 us; error 0 us; skew 500 ppm
I20260806 03:41:42.743897 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:42.746855 16369 webserver.cc:460] Webserver started at http://127.15.252.126:35243/ using document root <none> and password file <none>
I20260806 03:41:42.747361 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:42.747515 16369 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260806 03:41:42.751545 16369 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 03:41:42.755510 18579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:42.756562 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 03:41:42.756927 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-0-root
uuid: "30063023a8844495837679314257b563"
format_stamp: "Formatted at 2026-08-06 03:41:36 on dist-test-slave-csg5"
I20260806 03:41:42.757319 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:42.801388 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:42.802603 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:42.855135 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.126:42845
I20260806 03:41:42.855247 18631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.126:42845 every 8 connection(s)
I20260806 03:41:42.865443 18632 sys_catalog.cc:263] Verifying existing consensus state
I20260806 03:41:42.873001 18632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563: Bootstrap starting.
I20260806 03:41:42.902755 18305 heartbeater.cc:347] Connected to a master server at 127.15.252.126:42845
I20260806 03:41:42.908994 18142 heartbeater.cc:347] Connected to a master server at 127.15.252.126:42845
I20260806 03:41:42.926218 18224 heartbeater.cc:347] Connected to a master server at 127.15.252.126:42845
I20260806 03:41:42.955479 18632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=0 applied=17 ignored=0} inserts{seen=7 ignored=0} mutations{seen=13 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:41:42.956691 18632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563: Bootstrap complete.
I20260806 03:41:42.959995 18632 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:42.960875 18632 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30063023a8844495837679314257b563, State: Initialized, Role: FOLLOWER
I20260806 03:41:42.961897 16369 master.cc:537] Master@127.15.252.124:39623 shutting down...
I20260806 03:41:42.961658 18632 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 2.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:42.965585 18638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:42.967216 18638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:42.969265 18632 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:41:42.976899 18606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 2 FOLLOWER]: Advancing to term 3
I20260806 03:41:43.030813 18651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:43.032052 18651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:43.032487 16369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 3 LEADER]: Raft consensus shutting down.
I20260806 03:41:43.033341 18638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0591ed664d442fdaebaa2cd64c84b9f. Latest consensus state: current_term: 3 leader_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:43.033641 18652 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 03:41:43.033689 16369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:43.034169 18638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:43.034469 16369 tablet_replica.cc:330] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f: stopping tablet replica
I20260806 03:41:43.057301 16369 master.cc:552] Master@127.15.252.124:39623 shutdown complete.
I20260806 03:41:43.081748 18596 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "491cdd4ce26949848b317caa81d5279d" instance_seqno: 1785987697259302) as {username='slave'} at 127.0.0.1:34422; Asking this server to re-register.
I20260806 03:41:43.083768 18224 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:43.083693 18595 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" instance_seqno: 1785987697497837) as {username='slave'} at 127.0.0.1:34410; Asking this server to re-register.
I20260806 03:41:43.085477 18305 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:43.088526 18595 ts_manager.cc:194] Registered new tserver with Master: 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:43.090027 18596 ts_manager.cc:194] Registered new tserver with Master: 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:43.091717 18595 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "4abe04524a7046018757fcdfd13b815d" instance_seqno: 1785987697088220) as {username='slave'} at 127.0.0.1:34414; Asking this server to re-register.
I20260806 03:41:43.094113 18142 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:43.097491 18595 ts_manager.cc:194] Registered new tserver with Master: 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231)
I20260806 03:41:43.169638 18653 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c0591ed664d442fdaebaa2cd64c84b9f)
I20260806 03:41:43.170346 18653 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:43.172251 18653 leader_election.cc:288] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 30063023a8844495837679314257b563 (127.15.252.126:42845), c0591ed664d442fdaebaa2cd64c84b9f (127.15.252.124:39623)
W20260806 03:41:43.179719 18482 leader_election.cc:334] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c0591ed664d442fdaebaa2cd64c84b9f (127.15.252.124:39623): Network error: Client connection negotiation failed: client connection to 127.15.252.124:39623: connect: Connection refused (error 111)
I20260806 03:41:43.192292 18606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a5ff5ca39d348fa837104d428536561" candidate_term: 4 candidate_status { last_received { term: 3 index: 18 } } ignore_live_leader: false dest_uuid: "30063023a8844495837679314257b563" is_pre_election: true
I20260806 03:41:43.194192 18638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c0591ed664d442fdaebaa2cd64c84b9f)
I20260806 03:41:43.194092 18481 leader_election.cc:302] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a5ff5ca39d348fa837104d428536561; no voters: 30063023a8844495837679314257b563, c0591ed664d442fdaebaa2cd64c84b9f
I20260806 03:41:43.194865 18638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:43.195305 18653 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260806 03:41:43.197229 18638 leader_election.cc:288] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 2a5ff5ca39d348fa837104d428536561 (127.15.252.125:34597), c0591ed664d442fdaebaa2cd64c84b9f (127.15.252.124:39623)
W20260806 03:41:43.218641 18582 leader_election.cc:334] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c0591ed664d442fdaebaa2cd64c84b9f (127.15.252.124:39623): Network error: Client connection negotiation failed: client connection to 127.15.252.124:39623: connect: Connection refused (error 111)
I20260806 03:41:43.226965 18509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30063023a8844495837679314257b563" candidate_term: 4 candidate_status { last_received { term: 3 index: 18 } } ignore_live_leader: false dest_uuid: "2a5ff5ca39d348fa837104d428536561" is_pre_election: true
I20260806 03:41:43.227638 18509 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30063023a8844495837679314257b563 in term 3.
I20260806 03:41:43.228912 18583 leader_election.cc:302] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a5ff5ca39d348fa837104d428536561, 30063023a8844495837679314257b563; no voters: c0591ed664d442fdaebaa2cd64c84b9f
I20260806 03:41:43.229808 18638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260806 03:41:43.230165 18638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c0591ed664d442fdaebaa2cd64c84b9f)
I20260806 03:41:43.230582 18638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 3 FOLLOWER]: Advancing to term 4
I20260806 03:41:43.235700 18638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:43.237969 18638 leader_election.cc:288] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [CANDIDATE]: Term 4 election: Requested vote from peers 2a5ff5ca39d348fa837104d428536561 (127.15.252.125:34597), c0591ed664d442fdaebaa2cd64c84b9f (127.15.252.124:39623)
I20260806 03:41:43.238978 18509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30063023a8844495837679314257b563" candidate_term: 4 candidate_status { last_received { term: 3 index: 18 } } ignore_live_leader: false dest_uuid: "2a5ff5ca39d348fa837104d428536561"
I20260806 03:41:43.239641 18509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 3 FOLLOWER]: Advancing to term 4
W20260806 03:41:43.243098 18582 leader_election.cc:334] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer c0591ed664d442fdaebaa2cd64c84b9f (127.15.252.124:39623): Network error: Client connection negotiation failed: client connection to 127.15.252.124:39623: connect: Connection refused (error 111)
I20260806 03:41:43.245978 18509 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30063023a8844495837679314257b563 in term 4.
I20260806 03:41:43.247246 18583 leader_election.cc:302] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a5ff5ca39d348fa837104d428536561, 30063023a8844495837679314257b563; no voters: c0591ed664d442fdaebaa2cd64c84b9f
I20260806 03:41:43.248138 18638 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 4 FOLLOWER]: Leader election won for term 4
I20260806 03:41:43.248692 18638 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 4 LEADER]: Becoming Leader. State: Replica: 30063023a8844495837679314257b563, State: Running, Role: LEADER
I20260806 03:41:43.249473 18638 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 3.18, Last appended by leader: 18, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:43.253582 18651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30063023a8844495837679314257b563. Latest consensus state: current_term: 4 leader_uuid: "30063023a8844495837679314257b563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:43.254220 18651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:43.255563 18663 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:41:43.260632 18663 catalog_manager.cc:669] Loaded metadata for table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe]
I20260806 03:41:43.265506 18663 tablet_loader.cc:96] loaded metadata for tablet 4f6ce5ed631f413d907292ee85373f54 (table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe])
I20260806 03:41:43.266738 18663 tablet_loader.cc:96] loaded metadata for tablet 5390b9a07b5f4cd8b2330fd0974c8821 (table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe])
I20260806 03:41:43.267839 18663 tablet_loader.cc:96] loaded metadata for tablet 6c23a83f61a74931bb2a5239d32e1cad (table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe])
I20260806 03:41:43.268846 18663 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:41:43.272555 18663 catalog_manager.cc:1259] Loaded cluster ID: 8225191a001148569992571dec22b329
I20260806 03:41:43.272866 18663 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:41:43.279367 18663 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:41:43.280089 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:41:43.285537 18663 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563: Loaded TSK: 0
I20260806 03:41:43.290551 18663 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 03:41:43.291121 18664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:43.295904 18665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:43.299489 18667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:43.300619 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:43.302992 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:43.303337 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:43.303570 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987703303553 us; error 0 us; skew 500 ppm
I20260806 03:41:43.304389 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:43.311816 16369 webserver.cc:460] Webserver started at http://127.15.252.124:41187/ using document root <none> and password file <none>
I20260806 03:41:43.312397 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:43.312621 16369 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260806 03:41:43.316874 16369 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 03:41:43.320616 18672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:43.321785 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 03:41:43.322178 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-2-root
uuid: "c0591ed664d442fdaebaa2cd64c84b9f"
format_stamp: "Formatted at 2026-08-06 03:41:36 on dist-test-slave-csg5"
I20260806 03:41:43.322572 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260806 03:41:43.346587 18582 consensus_peers.cc:487] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 -> Peer c0591ed664d442fdaebaa2cd64c84b9f (127.15.252.124:39623): Couldn't send request to peer c0591ed664d442fdaebaa2cd64c84b9f. Status: Network error: Client connection negotiation failed: client connection to 127.15.252.124:39623: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 03:41:43.349220 18509 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 4 FOLLOWER]: Refusing update from remote peer 30063023a8844495837679314257b563: Log matching property violated. Preceding OpId in replica: term: 3 index: 18. Preceding OpId from leader: term: 4 index: 19. (index mismatch)
I20260806 03:41:43.349766 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:43.350966 18651 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260806 03:41:43.351559 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:43.356694 18653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30063023a8844495837679314257b563. Latest consensus state: current_term: 4 leader_uuid: "30063023a8844495837679314257b563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:43.357432 18653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:43.360632 18638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "30063023a8844495837679314257b563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:43.361356 18638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:43.363626 18653 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "30063023a8844495837679314257b563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:43.364956 18653 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:43.405155 18305 heartbeater.cc:503] Master 127.15.252.126:42845 was elected leader, sending a full tablet report...
I20260806 03:41:43.408528 18224 heartbeater.cc:503] Master 127.15.252.126:42845 was elected leader, sending a full tablet report...
I20260806 03:41:43.411257 18142 heartbeater.cc:503] Master 127.15.252.126:42845 was elected leader, sending a full tablet report...
I20260806 03:41:43.428344 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.124:39623
I20260806 03:41:43.428656 18726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.124:39623 every 8 connection(s)
I20260806 03:41:43.438508 18727 sys_catalog.cc:263] Verifying existing consensus state
I20260806 03:41:43.445914 18727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f: Bootstrap starting.
I20260806 03:41:43.525161 18303 heartbeater.cc:347] Connected to a master server at 127.15.252.124:39623
I20260806 03:41:43.530771 18140 heartbeater.cc:347] Connected to a master server at 127.15.252.124:39623
I20260806 03:41:43.531855 18727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f: Bootstrap replayed 1/1 log segments. Stats: ops{read=18 overwritten=0 applied=18 ignored=0} inserts{seen=7 ignored=0} mutations{seen=13 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:41:43.532258 18222 heartbeater.cc:347] Connected to a master server at 127.15.252.124:39623
I20260806 03:41:43.532965 18727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f: Bootstrap complete.
I20260806 03:41:43.533388 16369 tablet_server.cc:178] TabletServer@127.15.252.65:0 shutting down...
I20260806 03:41:43.540014 18727 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:43.541018 18727 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0591ed664d442fdaebaa2cd64c84b9f, State: Initialized, Role: FOLLOWER
I20260806 03:41:43.541822 18727 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 18, Last appended: 3.18, Last appended by leader: 18, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:43.553944 18733 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:43.555150 18733 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:43.557132 18727 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:41:43.559739 18701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 3 FOLLOWER]: Advancing to term 4
I20260806 03:41:43.596441 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:43.599531 16369 tablet_replica.cc:330] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d: stopping tablet replica
I20260806 03:41:43.600737 16369 raft_consensus.cc:2229] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260806 03:41:43.604609 16369 raft_consensus.cc:2258] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:43.610522 16369 tablet_replica.cc:330] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d: stopping tablet replica
I20260806 03:41:43.611634 16369 raft_consensus.cc:2229] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260806 03:41:43.611348 18733 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30063023a8844495837679314257b563. Latest consensus state: current_term: 4 leader_uuid: "30063023a8844495837679314257b563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:43.612452 16369 raft_consensus.cc:2258] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:43.612542 18733 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:43.615720 18743 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 4 leader_uuid: "30063023a8844495837679314257b563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:43.616781 18743 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:43.617403 16369 tablet_replica.cc:330] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d: stopping tablet replica
I20260806 03:41:43.619753 16369 raft_consensus.cc:2229] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 2 LEADER]: Raft consensus shutting down.
I20260806 03:41:43.620903 16369 raft_consensus.cc:2258] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260806 03:41:43.622406 18160 consensus_peers.cc:487] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d -> Peer 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231): Couldn't send request to peer 4abe04524a7046018757fcdfd13b815d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260806 03:41:43.627439 18160 consensus_peers.cc:487] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d -> Peer 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231): Couldn't send request to peer 4abe04524a7046018757fcdfd13b815d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260806 03:41:43.634369 18748 catalog_manager.cc:1259] Loaded cluster ID: 8225191a001148569992571dec22b329
I20260806 03:41:43.634847 18748 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f: loading cluster ID for follower catalog manager: success
I20260806 03:41:43.640789 18748 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f: acquiring CA information for follower catalog manager: success
I20260806 03:41:43.645283 18748 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 03:41:43.647961 16369 tablet_server.cc:194] TabletServer@127.15.252.65:0 shutdown complete.
I20260806 03:41:43.652083 18691 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" instance_seqno: 1785987697497837) as {username='slave'} at 127.0.0.1:37946; Asking this server to re-register.
I20260806 03:41:43.653810 18303 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:43.657961 18691 ts_manager.cc:194] Registered new tserver with Master: 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:43.661235 18690 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "491cdd4ce26949848b317caa81d5279d" instance_seqno: 1785987697259302) as {username='slave'} at 127.0.0.1:37972; Asking this server to re-register.
I20260806 03:41:43.662588 18222 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:43.666923 18690 ts_manager.cc:194] Registered new tserver with Master: 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:43.668928 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:41:43.672421 18651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 4 leader_uuid: "30063023a8844495837679314257b563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:43.673504 18651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: This master's current role is: LEADER
W20260806 03:41:43.685046 18751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:43.698128 18753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:43.698287 16369 server_base.cc:1034] running on GCE node
W20260806 03:41:43.699601 18750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:43.704054 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:43.704394 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:43.704627 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987703704610 us; error 0 us; skew 500 ppm
I20260806 03:41:43.705425 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:43.708460 16369 webserver.cc:460] Webserver started at http://127.15.252.65:45261/ using document root <none> and password file <none>
I20260806 03:41:43.708961 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:43.709110 16369 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260806 03:41:43.712888 16369 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 03:41:43.716289 18759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:43.719135 16369 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260806 03:41:43.719558 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-0-root
uuid: "4abe04524a7046018757fcdfd13b815d"
format_stamp: "Formatted at 2026-08-06 03:41:36 on dist-test-slave-csg5"
I20260806 03:41:43.719954 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:43.726917 18761 raft_consensus.cc:482] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4abe04524a7046018757fcdfd13b815d)
I20260806 03:41:43.727632 18761 raft_consensus.cc:504] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:43.729950 18761 leader_election.cc:288] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:43.731048 18196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d" is_pre_election: true
W20260806 03:41:43.734354 18241 leader_election.cc:334] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231): Network error: Client connection negotiation failed: client connection to 127.15.252.65:35231: connect: Connection refused (error 111)
I20260806 03:41:43.734774 18241 leader_election.cc:302] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6eb7ccdcc221406296e866e74a37ed75; no voters: 491cdd4ce26949848b317caa81d5279d, 4abe04524a7046018757fcdfd13b815d
I20260806 03:41:43.735395 18761 raft_consensus.cc:2726] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260806 03:41:43.736230 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:43.737493 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:43.739100 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:43.742990 18769 ts_tablet_manager.cc:542] Loading tablet metadata (0/3 complete)
I20260806 03:41:43.763389 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (3 total tablets, 3 live tablets)
I20260806 03:41:43.763757 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.022s user 0.001s sys 0.001s
I20260806 03:41:43.764163 16369 ts_tablet_manager.cc:594] Registering tablets (0/3 complete)
I20260806 03:41:43.768671 18566 raft_consensus.cc:482] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4abe04524a7046018757fcdfd13b815d)
I20260806 03:41:43.769513 18769 tablet_bootstrap.cc:492] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d: Bootstrap starting.
I20260806 03:41:43.769236 18566 raft_consensus.cc:504] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:43.771414 18566 leader_election.cc:288] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:43.773039 18276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
I20260806 03:41:43.773693 18276 raft_consensus.cc:2454] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 491cdd4ce26949848b317caa81d5279d in term 2.
I20260806 03:41:43.775044 18159 leader_election.cc:302] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d, 6eb7ccdcc221406296e866e74a37ed75; no voters:
I20260806 03:41:43.775898 18566 raft_consensus.cc:2781] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260806 03:41:43.776121 18160 leader_election.cc:334] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231): Network error: Client connection negotiation failed: client connection to 127.15.252.65:35231: connect: Connection refused (error 111)
I20260806 03:41:43.776445 18566 raft_consensus.cc:482] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4abe04524a7046018757fcdfd13b815d)
I20260806 03:41:43.776991 18566 raft_consensus.cc:3037] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 2 FOLLOWER]: Advancing to term 3
I20260806 03:41:43.780732 16369 ts_tablet_manager.cc:610] Registered 3 tablets
I20260806 03:41:43.781062 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.017s user 0.015s sys 0.000s
I20260806 03:41:43.786026 18566 raft_consensus.cc:504] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:43.790055 18276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75"
I20260806 03:41:43.790779 18276 raft_consensus.cc:3037] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 2 FOLLOWER]: Advancing to term 3
W20260806 03:41:43.793825 18160 leader_election.cc:334] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231): Network error: Client connection negotiation failed: client connection to 127.15.252.65:35231: connect: Connection refused (error 111)
I20260806 03:41:43.804234 18276 raft_consensus.cc:2454] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 491cdd4ce26949848b317caa81d5279d in term 3.
I20260806 03:41:43.806504 18159 leader_election.cc:302] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d, 6eb7ccdcc221406296e866e74a37ed75; no voters: 4abe04524a7046018757fcdfd13b815d
I20260806 03:41:43.804915 18566 leader_election.cc:288] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 3 election: Requested vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:43.807546 18760 raft_consensus.cc:2781] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Leader election won for term 3
I20260806 03:41:43.808169 18760 raft_consensus.cc:686] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 3 LEADER]: Becoming Leader. State: Replica: 491cdd4ce26949848b317caa81d5279d, State: Running, Role: LEADER
I20260806 03:41:43.809134 18760 consensus_queue.cc:237] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:43.821589 18596 catalog_manager.cc:5495] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d reported cstate change: term changed from 2 to 3, leader changed from 4abe04524a7046018757fcdfd13b815d (127.15.252.65) to 491cdd4ce26949848b317caa81d5279d (127.15.252.66). New cstate: current_term: 3 leader_uuid: "491cdd4ce26949848b317caa81d5279d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } health_report { overall_health: UNKNOWN } } }
I20260806 03:41:43.893253 18769 tablet.cc:1722] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 03:41:43.894310 18769 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20260806 03:41:43.896014 18769 tablet_bootstrap.cc:492] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=276 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:41:43.897099 18769 tablet_bootstrap.cc:492] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d: Bootstrap complete.
I20260806 03:41:43.909322 18769 ts_tablet_manager.cc:1397] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d: Time spent bootstrapping tablet: real 0.140s user 0.105s sys 0.028s
I20260806 03:41:43.912477 18769 raft_consensus.cc:348] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:43.913455 18769 raft_consensus.cc:729] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abe04524a7046018757fcdfd13b815d, State: Initialized, Role: FOLLOWER
I20260806 03:41:43.914191 18769 consensus_queue.cc:260] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:43.918401 18769 ts_tablet_manager.cc:1428] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d: Time spent starting tablet: real 0.009s user 0.010s sys 0.000s
I20260806 03:41:43.919457 18769 tablet_bootstrap.cc:492] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d: Bootstrap starting.
W20260806 03:41:43.920414 18160 consensus_peers.cc:487] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d -> Peer 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231): Couldn't send request to peer 4abe04524a7046018757fcdfd13b815d. Status: Network error: Client connection negotiation failed: client connection to 127.15.252.65:35231: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 03:41:43.922438 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.65:35231
I20260806 03:41:43.922679 18835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.65:35231 every 8 connection(s)
I20260806 03:41:43.936740 18276 raft_consensus.cc:1261] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Refusing update from remote peer 491cdd4ce26949848b317caa81d5279d: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260806 03:41:43.951756 18760 consensus_queue.cc:1035] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260806 03:41:44.032117 18807 raft_consensus.cc:3037] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 2 FOLLOWER]: Advancing to term 3
I20260806 03:41:44.040058 18837 heartbeater.cc:347] Connected to a master server at 127.15.252.125:34597
I20260806 03:41:44.040786 18837 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:44.041929 18837 heartbeater.cc:511] Master 127.15.252.125:34597 requested a full tablet report, sending...
I20260806 03:41:44.040030 18836 heartbeater.cc:347] Connected to a master server at 127.15.252.124:39623
I20260806 03:41:44.043370 18836 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:44.044643 18836 heartbeater.cc:511] Master 127.15.252.124:39623 requested a full tablet report, sending...
I20260806 03:41:44.064441 18497 ts_manager.cc:194] Re-registered known tserver with Master: 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231)
W20260806 03:41:44.072831 18160 consensus_peers.cc:487] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d -> Peer 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231): Couldn't send request to peer 4abe04524a7046018757fcdfd13b815d. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260806 03:41:44.082450 18838 heartbeater.cc:347] Connected to a master server at 127.15.252.126:42845
I20260806 03:41:44.083022 18838 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:44.084247 18838 heartbeater.cc:511] Master 127.15.252.126:42845 requested a full tablet report, sending...
I20260806 03:41:44.092442 18596 ts_manager.cc:194] Re-registered known tserver with Master: 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231)
I20260806 03:41:44.095753 18596 catalog_manager.cc:5587] TS 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231) does not have the latest schema for tablet 4f6ce5ed631f413d907292ee85373f54 (table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe]). Expected version 1 got 0
I20260806 03:41:44.095991 18690 ts_manager.cc:194] Registered new tserver with Master: 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231)
I20260806 03:41:44.103542 18596 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34452
I20260806 03:41:44.110801 18838 heartbeater.cc:503] Master 127.15.252.126:42845 was elected leader, sending a full tablet report...
I20260806 03:41:44.112893 18769 tablet.cc:1722] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 03:41:44.113631 18769 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20260806 03:41:44.114992 18769 tablet_bootstrap.cc:492] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=260 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:41:44.115978 18769 tablet_bootstrap.cc:492] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d: Bootstrap complete.
I20260806 03:41:44.118104 18596 catalog_manager.cc:5587] TS 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231) does not have the latest schema for tablet 4f6ce5ed631f413d907292ee85373f54 (table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe]). Expected version 1 got 0
I20260806 03:41:44.124848 18769 ts_tablet_manager.cc:1397] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d: Time spent bootstrapping tablet: real 0.206s user 0.120s sys 0.050s
I20260806 03:41:44.127014 18769 raft_consensus.cc:348] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:44.127703 18769 raft_consensus.cc:729] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abe04524a7046018757fcdfd13b815d, State: Initialized, Role: FOLLOWER
I20260806 03:41:44.128346 18769 consensus_queue.cc:260] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:44.130024 18769 ts_tablet_manager.cc:1428] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 03:41:44.131042 18769 tablet_bootstrap.cc:492] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d: Bootstrap starting.
I20260806 03:41:44.196786 18769 tablet.cc:1722] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 03:41:44.197582 18769 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20260806 03:41:44.198534 18769 tablet_bootstrap.cc:492] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=266 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:41:44.199429 18769 tablet_bootstrap.cc:492] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d: Bootstrap complete.
I20260806 03:41:44.206058 18769 ts_tablet_manager.cc:1397] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d: Time spent bootstrapping tablet: real 0.075s user 0.064s sys 0.006s
I20260806 03:41:44.207878 18769 raft_consensus.cc:348] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:44.208388 18769 raft_consensus.cc:729] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4abe04524a7046018757fcdfd13b815d, State: Initialized, Role: FOLLOWER
I20260806 03:41:44.208951 18769 consensus_queue.cc:260] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:44.211225 18769 ts_tablet_manager.cc:1428] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260806 03:41:44.217367 16369 tablet_server.cc:178] TabletServer@127.15.252.66:0 shutting down...
I20260806 03:41:44.284135 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:44.284982 16369 tablet_replica.cc:330] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d: stopping tablet replica
I20260806 03:41:44.285712 16369 raft_consensus.cc:2229] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 3 LEADER]: Raft consensus shutting down.
I20260806 03:41:44.286710 16369 raft_consensus.cc:2258] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:44.289137 16369 tablet_replica.cc:330] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d: stopping tablet replica
I20260806 03:41:44.289649 16369 raft_consensus.cc:2229] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 3 LEADER]: Raft consensus shutting down.
I20260806 03:41:44.290447 16369 raft_consensus.cc:2258] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:44.292694 16369 tablet_replica.cc:330] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d: stopping tablet replica
I20260806 03:41:44.293231 16369 raft_consensus.cc:2229] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 3 LEADER]: Raft consensus shutting down.
I20260806 03:41:44.294147 16369 raft_consensus.cc:2258] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:44.318756 16369 tablet_server.cc:194] TabletServer@127.15.252.66:0 shutdown complete.
I20260806 03:41:44.335052 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:44.343951 18856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:44.344192 18858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:44.345465 18860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:44.348278 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:44.353652 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:44.353971 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:44.354185 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987704354171 us; error 0 us; skew 500 ppm
I20260806 03:41:44.354863 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:44.357966 16369 webserver.cc:460] Webserver started at http://127.15.252.66:34677/ using document root <none> and password file <none>
I20260806 03:41:44.358587 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:44.358868 16369 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260806 03:41:44.362748 16369 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 03:41:44.366043 18865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:44.367053 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 03:41:44.367373 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-1-root
uuid: "491cdd4ce26949848b317caa81d5279d"
format_stamp: "Formatted at 2026-08-06 03:41:37 on dist-test-slave-csg5"
I20260806 03:41:44.367660 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:44.387542 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:44.388814 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:44.390161 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:44.396287 18761 raft_consensus.cc:482] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 491cdd4ce26949848b317caa81d5279d)
I20260806 03:41:44.396883 18761 raft_consensus.cc:504] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:44.398967 18761 leader_election.cc:288] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729), 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231)
I20260806 03:41:44.417788 18761 raft_consensus.cc:482] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 491cdd4ce26949848b317caa81d5279d)
I20260806 03:41:44.418573 18761 raft_consensus.cc:504] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
W20260806 03:41:44.419265 18239 leader_election.cc:334] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729): Network error: Client connection negotiation failed: client connection to 127.15.252.66:43729: connect: Connection refused (error 111)
I20260806 03:41:44.420749 18761 leader_election.cc:288] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:44.423158 18808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:44.423465 18807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:44.425361 18241 leader_election.cc:302] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6eb7ccdcc221406296e866e74a37ed75; no voters: 491cdd4ce26949848b317caa81d5279d, 4abe04524a7046018757fcdfd13b815d
I20260806 03:41:44.426299 18761 raft_consensus.cc:2726] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260806 03:41:44.426332 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (3 total tablets, 3 live tablets)
I20260806 03:41:44.427031 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.034s user 0.002s sys 0.000s
W20260806 03:41:44.427218 18239 leader_election.cc:334] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729): Network error: Client connection negotiation failed: client connection to 127.15.252.66:43729: connect: Connection refused (error 111)
I20260806 03:41:44.427644 18239 leader_election.cc:302] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6eb7ccdcc221406296e866e74a37ed75; no voters: 491cdd4ce26949848b317caa81d5279d, 4abe04524a7046018757fcdfd13b815d
I20260806 03:41:44.428284 18761 raft_consensus.cc:2726] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260806 03:41:44.432291 18872 tablet_bootstrap.cc:492] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d: Bootstrap starting.
I20260806 03:41:44.443075 16369 ts_tablet_manager.cc:610] Registered 3 tablets
I20260806 03:41:44.443413 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.016s user 0.015s sys 0.000s
I20260806 03:41:44.448010 18831 raft_consensus.cc:482] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 491cdd4ce26949848b317caa81d5279d)
I20260806 03:41:44.448551 18831 raft_consensus.cc:504] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:44.450526 18878 raft_consensus.cc:482] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 491cdd4ce26949848b317caa81d5279d)
I20260806 03:41:44.451104 18878 raft_consensus.cc:504] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:44.454073 18831 leader_election.cc:288] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:44.454432 18878 leader_election.cc:288] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:44.459945 18878 raft_consensus.cc:482] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 491cdd4ce26949848b317caa81d5279d)
I20260806 03:41:44.460669 18878 raft_consensus.cc:504] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:44.464213 18878 leader_election.cc:288] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363), 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
W20260806 03:41:44.475751 18763 leader_election.cc:334] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729): Network error: Client connection negotiation failed: client connection to 127.15.252.66:43729: connect: Connection refused (error 111)
I20260806 03:41:44.477375 18761 raft_consensus.cc:482] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 491cdd4ce26949848b317caa81d5279d)
I20260806 03:41:44.479094 18761 raft_consensus.cc:504] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
W20260806 03:41:44.484054 18763 leader_election.cc:334] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729): Network error: Client connection negotiation failed: client connection to 127.15.252.66:43729: connect: Connection refused (error 111)
W20260806 03:41:44.484783 18763 leader_election.cc:334] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729): Network error: Client connection negotiation failed: client connection to 127.15.252.66:43729: connect: Connection refused (error 111)
I20260806 03:41:44.485903 18807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:44.487000 18807 raft_consensus.cc:2454] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6eb7ccdcc221406296e866e74a37ed75 in term 3.
I20260806 03:41:44.489343 18241 leader_election.cc:302] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75; no voters:
I20260806 03:41:44.496447 18761 leader_election.cc:288] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:44.497262 18761 raft_consensus.cc:2781] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260806 03:41:44.497692 18761 raft_consensus.cc:482] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 491cdd4ce26949848b317caa81d5279d)
I20260806 03:41:44.497525 18276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
I20260806 03:41:44.497856 18274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
I20260806 03:41:44.498744 18274 raft_consensus.cc:2454] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4abe04524a7046018757fcdfd13b815d in term 3.
I20260806 03:41:44.499426 18275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
I20260806 03:41:44.500360 18275 raft_consensus.cc:2454] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4abe04524a7046018757fcdfd13b815d in term 3.
W20260806 03:41:44.503957 18239 leader_election.cc:334] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729): Network error: Client connection negotiation failed: client connection to 127.15.252.66:43729: connect: Connection refused (error 111)
I20260806 03:41:44.505558 18761 raft_consensus.cc:3037] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Advancing to term 4
I20260806 03:41:44.507114 18764 leader_election.cc:302] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75; no voters: 491cdd4ce26949848b317caa81d5279d
I20260806 03:41:44.508389 18878 raft_consensus.cc:2781] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260806 03:41:44.508842 18878 raft_consensus.cc:482] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Starting leader election (detected failure of leader 491cdd4ce26949848b317caa81d5279d)
I20260806 03:41:44.508944 18764 leader_election.cc:302] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75; no voters: 491cdd4ce26949848b317caa81d5279d
I20260806 03:41:44.509302 18878 raft_consensus.cc:3037] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Advancing to term 4
I20260806 03:41:44.510146 18831 raft_consensus.cc:2781] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260806 03:41:44.510636 18831 raft_consensus.cc:482] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Starting leader election (detected failure of leader 491cdd4ce26949848b317caa81d5279d)
I20260806 03:41:44.511147 18831 raft_consensus.cc:3037] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Advancing to term 4
I20260806 03:41:44.515007 18761 raft_consensus.cc:504] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:44.516399 18276 raft_consensus.cc:2379] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4abe04524a7046018757fcdfd13b815d in current term 4: Already voted for candidate 6eb7ccdcc221406296e866e74a37ed75 in this term.
I20260806 03:41:44.518486 18878 raft_consensus.cc:504] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:44.523232 18807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d"
I20260806 03:41:44.523773 18878 leader_election.cc:288] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 4 election: Requested vote from peers 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:44.524004 18807 raft_consensus.cc:3037] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 3 FOLLOWER]: Advancing to term 4
W20260806 03:41:44.526816 18763 leader_election.cc:334] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729): Network error: Client connection negotiation failed: client connection to 127.15.252.66:43729: connect: Connection refused (error 111)
I20260806 03:41:44.530287 18831 raft_consensus.cc:504] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:44.531358 18761 leader_election.cc:288] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 4 election: Requested vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:44.531970 18764 leader_election.cc:302] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d; no voters: 491cdd4ce26949848b317caa81d5279d, 6eb7ccdcc221406296e866e74a37ed75
I20260806 03:41:44.535292 18276 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75"
I20260806 03:41:44.536024 18276 raft_consensus.cc:3037] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Advancing to term 4
I20260806 03:41:44.537573 18807 raft_consensus.cc:2454] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6eb7ccdcc221406296e866e74a37ed75 in term 4.
W20260806 03:41:44.538911 18763 leader_election.cc:334] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729): Network error: Client connection negotiation failed: client connection to 127.15.252.66:43729: connect: Connection refused (error 111)
I20260806 03:41:44.539311 18878 raft_consensus.cc:2726] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260806 03:41:44.539613 18831 leader_election.cc:288] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 4 election: Requested vote from peers 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363), 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:44.539932 18241 leader_election.cc:302] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75; no voters:
I20260806 03:41:44.539134 18275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75"
I20260806 03:41:44.540926 18761 raft_consensus.cc:2781] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Leader election won for term 4
I20260806 03:41:44.541391 18275 raft_consensus.cc:3037] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 3 FOLLOWER]: Advancing to term 4
I20260806 03:41:44.541785 18761 raft_consensus.cc:686] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 4 LEADER]: Becoming Leader. State: Replica: 6eb7ccdcc221406296e866e74a37ed75, State: Running, Role: LEADER
I20260806 03:41:44.542773 18761 consensus_queue.cc:237] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
W20260806 03:41:44.550565 18239 leader_election.cc:334] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729): Network error: Client connection negotiation failed: client connection to 127.15.252.66:43729: connect: Connection refused (error 111)
I20260806 03:41:44.571338 18275 raft_consensus.cc:2454] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4abe04524a7046018757fcdfd13b815d in term 4.
I20260806 03:41:44.569518 18276 raft_consensus.cc:2454] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4abe04524a7046018757fcdfd13b815d in term 4.
I20260806 03:41:44.573381 18764 leader_election.cc:302] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75; no voters: 491cdd4ce26949848b317caa81d5279d
I20260806 03:41:44.574440 18878 raft_consensus.cc:2781] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 4 FOLLOWER]: Leader election won for term 4
I20260806 03:41:44.574908 18764 leader_election.cc:302] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75; no voters: 491cdd4ce26949848b317caa81d5279d
I20260806 03:41:44.575121 18878 raft_consensus.cc:686] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 4 LEADER]: Becoming Leader. State: Replica: 4abe04524a7046018757fcdfd13b815d, State: Running, Role: LEADER
I20260806 03:41:44.576166 18831 raft_consensus.cc:2781] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 4 FOLLOWER]: Leader election won for term 4
I20260806 03:41:44.576452 18878 consensus_queue.cc:237] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:44.576593 18831 raft_consensus.cc:686] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 4 LEADER]: Becoming Leader. State: Replica: 4abe04524a7046018757fcdfd13b815d, State: Running, Role: LEADER
I20260806 03:41:44.579267 18831 consensus_queue.cc:237] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:44.585094 18872 tablet.cc:1722] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 03:41:44.585947 18872 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20260806 03:41:44.588701 18872 tablet_bootstrap.cc:492] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=276 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:41:44.589601 18872 tablet_bootstrap.cc:492] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d: Bootstrap complete.
I20260806 03:41:44.591915 18595 catalog_manager.cc:5495] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d reported cstate change: term changed from 3 to 4, leader changed from 491cdd4ce26949848b317caa81d5279d (127.15.252.66) to 4abe04524a7046018757fcdfd13b815d (127.15.252.65). New cstate: current_term: 4 leader_uuid: "4abe04524a7046018757fcdfd13b815d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } health_report { overall_health: HEALTHY } } }
I20260806 03:41:44.606999 18872 ts_tablet_manager.cc:1397] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d: Time spent bootstrapping tablet: real 0.175s user 0.109s sys 0.024s
I20260806 03:41:44.609875 18872 raft_consensus.cc:348] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:44.610638 18872 raft_consensus.cc:729] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 491cdd4ce26949848b317caa81d5279d, State: Initialized, Role: FOLLOWER
I20260806 03:41:44.611447 18872 consensus_queue.cc:260] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:44.615549 18596 catalog_manager.cc:5495] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 reported cstate change: term changed from 3 to 4, leader changed from 491cdd4ce26949848b317caa81d5279d (127.15.252.66) to 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67). New cstate: current_term: 4 leader_uuid: "6eb7ccdcc221406296e866e74a37ed75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } health_report { overall_health: HEALTHY } } }
I20260806 03:41:44.619570 18872 ts_tablet_manager.cc:1428] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d: Time spent starting tablet: real 0.012s user 0.007s sys 0.001s
W20260806 03:41:44.619537 18764 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.15.252.67:35363, user_credentials={real_user=slave}} blocked reactor thread for 44716.7us
I20260806 03:41:44.620867 18872 tablet_bootstrap.cc:492] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d: Bootstrap starting.
I20260806 03:41:44.642828 18807 raft_consensus.cc:1261] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 4 FOLLOWER]: Refusing update from remote peer 6eb7ccdcc221406296e866e74a37ed75: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260806 03:41:44.645371 18761 consensus_queue.cc:1035] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260806 03:41:44.662493 18239 consensus_peers.cc:487] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 -> Peer 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729): Couldn't send request to peer 491cdd4ce26949848b317caa81d5279d. Status: Network error: Client connection negotiation failed: client connection to 127.15.252.66:43729: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 03:41:44.691912 18275 raft_consensus.cc:1261] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Refusing update from remote peer 4abe04524a7046018757fcdfd13b815d: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260806 03:41:44.714275 18275 raft_consensus.cc:1261] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Refusing update from remote peer 4abe04524a7046018757fcdfd13b815d: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260806 03:41:44.719681 18891 consensus_queue.cc:1035] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260806 03:41:44.721912 18878 consensus_queue.cc:1035] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260806 03:41:44.740829 18763 consensus_peers.cc:487] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d -> Peer 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729): Couldn't send request to peer 491cdd4ce26949848b317caa81d5279d. Status: Network error: Client connection negotiation failed: client connection to 127.15.252.66:43729: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 03:41:44.749256 18872 tablet.cc:1722] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 03:41:44.750015 18872 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20260806 03:41:44.751179 18872 tablet_bootstrap.cc:492] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=260 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:41:44.752079 18872 tablet_bootstrap.cc:492] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d: Bootstrap complete.
I20260806 03:41:44.788813 18872 ts_tablet_manager.cc:1397] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d: Time spent bootstrapping tablet: real 0.168s user 0.115s sys 0.009s
I20260806 03:41:44.761046 18596 catalog_manager.cc:5495] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d reported cstate change: term changed from 3 to 4, leader changed from 491cdd4ce26949848b317caa81d5279d (127.15.252.66) to 4abe04524a7046018757fcdfd13b815d (127.15.252.65). New cstate: current_term: 4 leader_uuid: "4abe04524a7046018757fcdfd13b815d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } health_report { overall_health: UNKNOWN } } }
I20260806 03:41:44.791293 18872 raft_consensus.cc:348] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:44.792124 18872 raft_consensus.cc:729] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 491cdd4ce26949848b317caa81d5279d, State: Initialized, Role: FOLLOWER
I20260806 03:41:44.792889 18872 consensus_queue.cc:260] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:44.795028 18872 ts_tablet_manager.cc:1428] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 03:41:44.795789 18872 tablet_bootstrap.cc:492] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d: Bootstrap starting.
W20260806 03:41:44.813391 18763 consensus_peers.cc:487] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d -> Peer 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729): Couldn't send request to peer 491cdd4ce26949848b317caa81d5279d. Status: Network error: Client connection negotiation failed: client connection to 127.15.252.66:43729: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 03:41:44.859161 18899 raft_consensus.cc:482] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:41:44.859773 18899 raft_consensus.cc:504] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:44.866694 18899 leader_election.cc:288] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:44.886727 18275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
I20260806 03:41:44.896991 18807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:44.898965 18869 leader_election.cc:302] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d; no voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75
I20260806 03:41:44.900009 18899 raft_consensus.cc:2726] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260806 03:41:44.920193 18872 tablet.cc:1722] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 03:41:44.921090 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.66:43729
I20260806 03:41:44.921353 18872 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20260806 03:41:44.921278 18960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.66:43729 every 8 connection(s)
I20260806 03:41:44.922824 18872 tablet_bootstrap.cc:492] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=266 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:41:44.923812 18872 tablet_bootstrap.cc:492] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d: Bootstrap complete.
I20260806 03:41:44.938215 18872 ts_tablet_manager.cc:1397] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d: Time spent bootstrapping tablet: real 0.143s user 0.106s sys 0.011s
I20260806 03:41:44.941943 18872 raft_consensus.cc:348] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:44.943193 18872 raft_consensus.cc:729] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 491cdd4ce26949848b317caa81d5279d, State: Initialized, Role: FOLLOWER
I20260806 03:41:44.944768 18872 consensus_queue.cc:260] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:44.947854 18872 ts_tablet_manager.cc:1428] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260806 03:41:44.981637 18964 heartbeater.cc:347] Connected to a master server at 127.15.252.125:34597
I20260806 03:41:44.982117 18964 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:44.983152 18964 heartbeater.cc:511] Master 127.15.252.125:34597 requested a full tablet report, sending...
I20260806 03:41:44.989923 18496 ts_manager.cc:194] Re-registered known tserver with Master: 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:45.004416 18932 raft_consensus.cc:3037] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Advancing to term 4
I20260806 03:41:45.007942 18931 raft_consensus.cc:3037] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Advancing to term 4
I20260806 03:41:45.017340 18961 heartbeater.cc:347] Connected to a master server at 127.15.252.124:39623
I20260806 03:41:45.017853 18961 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:45.019111 18961 heartbeater.cc:511] Master 127.15.252.124:39623 requested a full tablet report, sending...
I20260806 03:41:45.024561 18928 raft_consensus.cc:3037] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 3 FOLLOWER]: Advancing to term 4
I20260806 03:41:45.030782 18691 ts_manager.cc:194] Re-registered known tserver with Master: 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:45.042490 18966 heartbeater.cc:347] Connected to a master server at 127.15.252.126:42845
I20260806 03:41:45.042970 18966 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:45.044039 18966 heartbeater.cc:511] Master 127.15.252.126:42845 requested a full tablet report, sending...
I20260806 03:41:45.048255 18596 ts_manager.cc:194] Re-registered known tserver with Master: 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:45.054803 18596 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34460
I20260806 03:41:45.065575 18966 heartbeater.cc:503] Master 127.15.252.126:42845 was elected leader, sending a full tablet report...
I20260806 03:41:45.208909 16369 tablet_server.cc:178] TabletServer@127.15.252.67:0 shutting down...
W20260806 03:41:45.251667 18764 consensus_peers.cc:487] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d -> Peer 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363): Couldn't send request to peer 6eb7ccdcc221406296e866e74a37ed75. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260806 03:41:45.252563 18764 consensus_peers.cc:487] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d -> Peer 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363): Couldn't send request to peer 6eb7ccdcc221406296e866e74a37ed75. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260806 03:41:45.257928 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:45.258841 16369 tablet_replica.cc:330] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75: stopping tablet replica
I20260806 03:41:45.259464 16369 raft_consensus.cc:2229] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260806 03:41:45.259992 16369 raft_consensus.cc:2258] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:45.262477 16369 tablet_replica.cc:330] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75: stopping tablet replica
I20260806 03:41:45.263074 16369 raft_consensus.cc:2229] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260806 03:41:45.263557 16369 raft_consensus.cc:2258] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:45.266013 16369 tablet_replica.cc:330] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75: stopping tablet replica
I20260806 03:41:45.266623 16369 raft_consensus.cc:2229] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 4 LEADER]: Raft consensus shutting down.
I20260806 03:41:45.267834 16369 raft_consensus.cc:2258] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:45.294848 16369 tablet_server.cc:194] TabletServer@127.15.252.67:0 shutdown complete.
I20260806 03:41:45.309674 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:45.316649 18975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:45.317329 18976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:45.319011 18978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:45.320699 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:45.327116 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:45.327445 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:45.327687 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987705327670 us; error 0 us; skew 500 ppm
I20260806 03:41:45.328585 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:45.344002 16369 webserver.cc:460] Webserver started at http://127.15.252.67:45789/ using document root <none> and password file <none>
I20260806 03:41:45.344726 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:45.344970 16369 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260806 03:41:45.354058 16369 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.006s sys 0.001s
I20260806 03:41:45.358927 18983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:45.360008 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 03:41:45.360402 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-2-root
uuid: "6eb7ccdcc221406296e866e74a37ed75"
format_stamp: "Formatted at 2026-08-06 03:41:37 on dist-test-slave-csg5"
I20260806 03:41:45.360777 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:45.363178 18831 raft_consensus.cc:482] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6eb7ccdcc221406296e866e74a37ed75)
I20260806 03:41:45.364002 18831 raft_consensus.cc:504] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:45.365741 18831 leader_election.cc:288] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:45.367010 18928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d" is_pre_election: true
W20260806 03:41:45.369904 18764 leader_election.cc:334] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363): Network error: Client connection negotiation failed: client connection to 127.15.252.67:35363: connect: Connection refused (error 111)
I20260806 03:41:45.370411 18764 leader_election.cc:302] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d; no voters: 491cdd4ce26949848b317caa81d5279d, 6eb7ccdcc221406296e866e74a37ed75
I20260806 03:41:45.371685 18831 raft_consensus.cc:2726] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260806 03:41:45.378376 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:45.379832 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:45.381402 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:45.384845 18990 ts_tablet_manager.cc:542] Loading tablet metadata (0/3 complete)
I20260806 03:41:45.402328 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (3 total tablets, 3 live tablets)
I20260806 03:41:45.402647 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.019s user 0.002s sys 0.000s
I20260806 03:41:45.402980 16369 ts_tablet_manager.cc:594] Registering tablets (0/3 complete)
I20260806 03:41:45.407718 18990 tablet_bootstrap.cc:492] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75: Bootstrap starting.
I20260806 03:41:45.415630 16369 ts_tablet_manager.cc:610] Registered 3 tablets
I20260806 03:41:45.415954 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.013s user 0.012s sys 0.000s
I20260806 03:41:45.432518 18899 raft_consensus.cc:482] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6eb7ccdcc221406296e866e74a37ed75)
I20260806 03:41:45.433142 18899 raft_consensus.cc:504] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:45.435408 18899 leader_election.cc:288] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:45.438086 18807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:45.438900 18807 raft_consensus.cc:2454] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 491cdd4ce26949848b317caa81d5279d in term 4.
I20260806 03:41:45.448733 18869 leader_election.cc:302] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d, 4abe04524a7046018757fcdfd13b815d; no voters:
W20260806 03:41:45.459657 18868 leader_election.cc:334] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363): Network error: Client connection negotiation failed: client connection to 127.15.252.67:35363: connect: Connection refused (error 111)
I20260806 03:41:45.465657 18899 raft_consensus.cc:2781] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260806 03:41:45.466316 18899 raft_consensus.cc:482] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6eb7ccdcc221406296e866e74a37ed75)
I20260806 03:41:45.466928 18899 raft_consensus.cc:3037] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 4 FOLLOWER]: Advancing to term 5
I20260806 03:41:45.477922 18899 raft_consensus.cc:504] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:45.481240 18899 leader_election.cc:288] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 5 election: Requested vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:45.481534 18807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d"
I20260806 03:41:45.482244 18807 raft_consensus.cc:3037] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 4 FOLLOWER]: Advancing to term 5
W20260806 03:41:45.492520 18868 leader_election.cc:334] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363): Network error: Client connection negotiation failed: client connection to 127.15.252.67:35363: connect: Connection refused (error 111)
I20260806 03:41:45.495412 18807 raft_consensus.cc:2454] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 491cdd4ce26949848b317caa81d5279d in term 5.
I20260806 03:41:45.497025 18869 leader_election.cc:302] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d, 4abe04524a7046018757fcdfd13b815d; no voters: 6eb7ccdcc221406296e866e74a37ed75
I20260806 03:41:45.498684 18899 raft_consensus.cc:2781] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 5 FOLLOWER]: Leader election won for term 5
I20260806 03:41:45.500712 18899 raft_consensus.cc:686] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 5 LEADER]: Becoming Leader. State: Replica: 491cdd4ce26949848b317caa81d5279d, State: Running, Role: LEADER
I20260806 03:41:45.502874 18899 consensus_queue.cc:237] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:45.516645 18990 tablet.cc:1722] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 03:41:45.517529 18990 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20260806 03:41:45.517393 18595 catalog_manager.cc:5495] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d reported cstate change: term changed from 4 to 5, leader changed from 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67) to 491cdd4ce26949848b317caa81d5279d (127.15.252.66). New cstate: current_term: 5 leader_uuid: "491cdd4ce26949848b317caa81d5279d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } health_report { overall_health: UNKNOWN } } }
I20260806 03:41:45.521843 18990 tablet_bootstrap.cc:492] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=276 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:41:45.522853 18990 tablet_bootstrap.cc:492] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75: Bootstrap complete.
I20260806 03:41:45.550459 18990 ts_tablet_manager.cc:1397] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75: Time spent bootstrapping tablet: real 0.143s user 0.085s sys 0.033s
I20260806 03:41:45.553485 18990 raft_consensus.cc:348] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:45.554549 18990 raft_consensus.cc:729] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eb7ccdcc221406296e866e74a37ed75, State: Initialized, Role: FOLLOWER
I20260806 03:41:45.555701 18990 consensus_queue.cc:260] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:45.574249 18990 ts_tablet_manager.cc:1428] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75: Time spent starting tablet: real 0.023s user 0.009s sys 0.013s
I20260806 03:41:45.575481 18990 tablet_bootstrap.cc:492] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75: Bootstrap starting.
W20260806 03:41:45.593849 18868 consensus_peers.cc:487] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d -> Peer 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363): Couldn't send request to peer 6eb7ccdcc221406296e866e74a37ed75. Status: Network error: Client connection negotiation failed: client connection to 127.15.252.67:35363: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 03:41:45.629142 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.67:35363
I20260806 03:41:45.630308 18807 raft_consensus.cc:1261] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 5 FOLLOWER]: Refusing update from remote peer 491cdd4ce26949848b317caa81d5279d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260806 03:41:45.629295 19062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.67:35363 every 8 connection(s)
I20260806 03:41:45.632270 18899 consensus_queue.cc:1035] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 03:41:45.722071 19064 heartbeater.cc:347] Connected to a master server at 127.15.252.125:34597
I20260806 03:41:45.722889 19064 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:45.723150 18990 tablet.cc:1722] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 03:41:45.724117 18990 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20260806 03:41:45.726574 18990 tablet_bootstrap.cc:492] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=260 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:41:45.727787 18990 tablet_bootstrap.cc:492] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75: Bootstrap complete.
I20260806 03:41:45.729338 19064 heartbeater.cc:511] Master 127.15.252.125:34597 requested a full tablet report, sending...
I20260806 03:41:45.734921 18494 ts_manager.cc:194] Re-registered known tserver with Master: 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
W20260806 03:41:45.740556 18764 consensus_peers.cc:487] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d -> Peer 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363): Couldn't send request to peer 6eb7ccdcc221406296e866e74a37ed75. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20260806 03:41:45.744736 19063 heartbeater.cc:347] Connected to a master server at 127.15.252.124:39623
I20260806 03:41:45.745239 19063 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:45.746186 19063 heartbeater.cc:511] Master 127.15.252.124:39623 requested a full tablet report, sending...
I20260806 03:41:45.747701 18990 ts_tablet_manager.cc:1397] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75: Time spent bootstrapping tablet: real 0.172s user 0.096s sys 0.021s
I20260806 03:41:45.751197 18691 ts_manager.cc:194] Re-registered known tserver with Master: 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:45.752899 19072 heartbeater.cc:347] Connected to a master server at 127.15.252.126:42845
I20260806 03:41:45.753324 19072 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:45.753764 18990 raft_consensus.cc:348] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:45.754302 19072 heartbeater.cc:511] Master 127.15.252.126:42845 requested a full tablet report, sending...
I20260806 03:41:45.755530 18990 raft_consensus.cc:729] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eb7ccdcc221406296e866e74a37ed75, State: Initialized, Role: FOLLOWER
I20260806 03:41:45.756313 18990 consensus_queue.cc:260] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:45.760257 18990 ts_tablet_manager.cc:1428] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75: Time spent starting tablet: real 0.012s user 0.004s sys 0.002s
I20260806 03:41:45.761094 18595 ts_manager.cc:194] Re-registered known tserver with Master: 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:45.761318 18990 tablet_bootstrap.cc:492] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75: Bootstrap starting.
I20260806 03:41:45.766906 18595 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34474
I20260806 03:41:45.771077 19072 heartbeater.cc:503] Master 127.15.252.126:42845 was elected leader, sending a full tablet report...
I20260806 03:41:45.778767 18595 catalog_manager.cc:5587] TS 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363) does not have the latest schema for tablet 6c23a83f61a74931bb2a5239d32e1cad (table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe]). Expected version 1 got 0
I20260806 03:41:45.851485 18990 tablet.cc:1722] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75: Alter schema from (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20260806 03:41:45.852324 18990 tablet_metadata.cc:623] Not flushing: waiting for 1 pins to be released.
I20260806 03:41:45.854743 18990 tablet_bootstrap.cc:492] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=266 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:41:45.855633 18990 tablet_bootstrap.cc:492] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75: Bootstrap complete.
I20260806 03:41:45.866834 18990 ts_tablet_manager.cc:1397] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75: Time spent bootstrapping tablet: real 0.106s user 0.092s sys 0.006s
I20260806 03:41:45.869346 18990 raft_consensus.cc:348] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:45.870239 18990 raft_consensus.cc:729] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eb7ccdcc221406296e866e74a37ed75, State: Initialized, Role: FOLLOWER
I20260806 03:41:45.875488 18990 consensus_queue.cc:260] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:45.878034 18990 ts_tablet_manager.cc:1428] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75: Time spent starting tablet: real 0.011s user 0.004s sys 0.000s
I20260806 03:41:46.083513 19083 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 30063023a8844495837679314257b563)
I20260806 03:41:46.084306 19083 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:46.086514 19083 leader_election.cc:288] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 30063023a8844495837679314257b563 (127.15.252.126:42845), c0591ed664d442fdaebaa2cd64c84b9f (127.15.252.124:39623)
I20260806 03:41:46.089032 18606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a5ff5ca39d348fa837104d428536561" candidate_term: 5 candidate_status { last_received { term: 4 index: 24 } } ignore_live_leader: false dest_uuid: "30063023a8844495837679314257b563" is_pre_election: true
I20260806 03:41:46.101198 18701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a5ff5ca39d348fa837104d428536561" candidate_term: 5 candidate_status { last_received { term: 4 index: 24 } } ignore_live_leader: false dest_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" is_pre_election: true
I20260806 03:41:46.101925 18701 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a5ff5ca39d348fa837104d428536561 in term 4.
I20260806 03:41:46.103410 18482 leader_election.cc:302] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a5ff5ca39d348fa837104d428536561, c0591ed664d442fdaebaa2cd64c84b9f; no voters: 30063023a8844495837679314257b563
I20260806 03:41:46.105248 19083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260806 03:41:46.105665 19083 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 30063023a8844495837679314257b563)
I20260806 03:41:46.106140 19083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 4 FOLLOWER]: Advancing to term 5
I20260806 03:41:46.112685 18878 raft_consensus.cc:482] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 491cdd4ce26949848b317caa81d5279d)
I20260806 03:41:46.113322 18878 raft_consensus.cc:504] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:46.113831 19083 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:46.117002 18878 leader_election.cc:288] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:46.116976 19083 leader_election.cc:288] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [CANDIDATE]: Term 5 election: Requested vote from peers 30063023a8844495837679314257b563 (127.15.252.126:42845), c0591ed664d442fdaebaa2cd64c84b9f (127.15.252.124:39623)
I20260806 03:41:46.117784 18606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a5ff5ca39d348fa837104d428536561" candidate_term: 5 candidate_status { last_received { term: 4 index: 24 } } ignore_live_leader: false dest_uuid: "30063023a8844495837679314257b563"
I20260806 03:41:46.121497 18701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a5ff5ca39d348fa837104d428536561" candidate_term: 5 candidate_status { last_received { term: 4 index: 24 } } ignore_live_leader: false dest_uuid: "c0591ed664d442fdaebaa2cd64c84b9f"
I20260806 03:41:46.122642 19037 raft_consensus.cc:482] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4abe04524a7046018757fcdfd13b815d)
I20260806 03:41:46.122931 18482 leader_election.cc:302] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a5ff5ca39d348fa837104d428536561; no voters: 30063023a8844495837679314257b563, c0591ed664d442fdaebaa2cd64c84b9f
I20260806 03:41:46.123257 19037 raft_consensus.cc:504] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:46.125859 19037 leader_election.cc:288] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:46.126906 19083 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260806 03:41:46.128088 19026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
I20260806 03:41:46.128830 19026 raft_consensus.cc:2454] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4abe04524a7046018757fcdfd13b815d in term 4.
I20260806 03:41:46.130089 18764 leader_election.cc:302] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75; no voters:
I20260806 03:41:46.131036 18878 raft_consensus.cc:2781] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260806 03:41:46.131769 18878 raft_consensus.cc:482] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 5 FOLLOWER]: Starting leader election (detected failure of leader 491cdd4ce26949848b317caa81d5279d)
I20260806 03:41:46.132221 18878 raft_consensus.cc:3037] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 5 FOLLOWER]: Advancing to term 6
I20260806 03:41:46.133496 19037 raft_consensus.cc:482] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4abe04524a7046018757fcdfd13b815d)
I20260806 03:41:46.137293 19037 raft_consensus.cc:504] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:46.140339 19037 leader_election.cc:288] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729), 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231)
I20260806 03:41:46.141541 18878 raft_consensus.cc:504] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:46.152192 19026 raft_consensus.cc:3037] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Advancing to term 5
I20260806 03:41:46.153038 18878 leader_election.cc:288] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 6 election: Requested vote from peers 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:46.153745 19027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75"
I20260806 03:41:46.156844 19021 raft_consensus.cc:482] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4abe04524a7046018757fcdfd13b815d)
I20260806 03:41:46.157492 19021 raft_consensus.cc:504] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:46.159466 18508 raft_consensus.cc:1226] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 5 FOLLOWER]: Rejecting Update request from peer 30063023a8844495837679314257b563 for earlier term 4. Current term is 5. Ops: []
I20260806 03:41:46.160164 19021 leader_election.cc:288] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363), 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231)
I20260806 03:41:46.160482 18899 raft_consensus.cc:482] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4abe04524a7046018757fcdfd13b815d)
I20260806 03:41:46.161051 18899 raft_consensus.cc:504] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:46.162926 19033 consensus_queue.cc:1046] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 }, Status: INVALID_TERM, Last received: 4.24, Next index: 25, Last known committed idx: 24, Time since last communication: 0.001s
I20260806 03:41:46.164011 19033 raft_consensus.cc:3032] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 4 LEADER]: Stepping down as leader of term 4
I20260806 03:41:46.164361 19033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 30063023a8844495837679314257b563, State: Running, Role: LEADER
I20260806 03:41:46.166541 18899 leader_election.cc:288] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
I20260806 03:41:46.167990 19027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
I20260806 03:41:46.168529 19025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75" is_pre_election: true
I20260806 03:41:46.168829 19027 raft_consensus.cc:2454] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 491cdd4ce26949848b317caa81d5279d in term 4.
I20260806 03:41:46.169262 19025 raft_consensus.cc:2454] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 491cdd4ce26949848b317caa81d5279d in term 4.
I20260806 03:41:46.171895 18931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d"
I20260806 03:41:46.175377 18868 leader_election.cc:302] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d, 6eb7ccdcc221406296e866e74a37ed75; no voters:
I20260806 03:41:46.177336 18763 leader_election.cc:302] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4abe04524a7046018757fcdfd13b815d; no voters: 491cdd4ce26949848b317caa81d5279d, 6eb7ccdcc221406296e866e74a37ed75
I20260806 03:41:46.178833 18878 raft_consensus.cc:2726] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260806 03:41:46.176323 18899 raft_consensus.cc:2781] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260806 03:41:46.181499 18899 raft_consensus.cc:482] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4abe04524a7046018757fcdfd13b815d)
I20260806 03:41:46.181922 18899 raft_consensus.cc:3037] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 4 FOLLOWER]: Advancing to term 5
I20260806 03:41:46.165155 19033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 4.24, Last appended by leader: 24, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:46.186103 18807 raft_consensus.cc:1226] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 6 FOLLOWER]: Rejecting Update request from peer 491cdd4ce26949848b317caa81d5279d for earlier term 5. Current term is 6. Ops: []
I20260806 03:41:46.186817 19033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 4 FOLLOWER]: Advancing to term 5
I20260806 03:41:46.187808 18807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:46.189597 18807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:46.191327 18868 leader_election.cc:302] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d, 6eb7ccdcc221406296e866e74a37ed75; no voters:
I20260806 03:41:46.192885 18932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "4abe04524a7046018757fcdfd13b815d" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d" is_pre_election: true
I20260806 03:41:46.193204 18807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:46.194743 18807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d" is_pre_election: true
I20260806 03:41:46.204890 18899 raft_consensus.cc:504] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:46.209046 18899 leader_election.cc:288] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 5 election: Requested vote from peers 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363), 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231)
I20260806 03:41:46.209735 18899 raft_consensus.cc:2781] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260806 03:41:46.209609 19021 consensus_queue.cc:1046] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260806 03:41:46.210577 19021 raft_consensus.cc:3032] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 5 LEADER]: Stepping down as leader of term 5
I20260806 03:41:46.210943 19021 raft_consensus.cc:729] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 491cdd4ce26949848b317caa81d5279d, State: Running, Role: LEADER
I20260806 03:41:46.210211 18899 raft_consensus.cc:482] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4abe04524a7046018757fcdfd13b815d)
I20260806 03:41:46.212229 19025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75"
I20260806 03:41:46.213040 19025 raft_consensus.cc:3037] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Advancing to term 5
I20260806 03:41:46.216125 19021 consensus_queue.cc:260] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:46.217934 19021 raft_consensus.cc:3037] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 5 FOLLOWER]: Advancing to term 6
I20260806 03:41:46.219889 19025 raft_consensus.cc:2454] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 491cdd4ce26949848b317caa81d5279d in term 5.
I20260806 03:41:46.220561 19027 raft_consensus.cc:1226] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 5 FOLLOWER]: Rejecting Update request from peer 4abe04524a7046018757fcdfd13b815d for earlier term 4. Current term is 5. Ops: []
I20260806 03:41:46.225571 18899 raft_consensus.cc:3037] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 4 FOLLOWER]: Advancing to term 5
I20260806 03:41:46.229769 18868 leader_election.cc:302] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d, 6eb7ccdcc221406296e866e74a37ed75; no voters:
I20260806 03:41:46.231032 19088 raft_consensus.cc:2781] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 5 FOLLOWER]: Leader election won for term 5
I20260806 03:41:46.231104 18807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4abe04524a7046018757fcdfd13b815d"
I20260806 03:41:46.231673 18928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f6ce5ed631f413d907292ee85373f54" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d" is_pre_election: true
I20260806 03:41:46.232405 19087 consensus_queue.cc:1046] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 03:41:46.231155 18932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d" is_pre_election: true
I20260806 03:41:46.234083 18985 leader_election.cc:302] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6eb7ccdcc221406296e866e74a37ed75; no voters: 491cdd4ce26949848b317caa81d5279d, 4abe04524a7046018757fcdfd13b815d
I20260806 03:41:46.235044 19085 raft_consensus.cc:2726] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260806 03:41:46.237210 19087 raft_consensus.cc:3032] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 4 LEADER]: Stepping down as leader of term 4
I20260806 03:41:46.237696 19087 raft_consensus.cc:729] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4abe04524a7046018757fcdfd13b815d, State: Running, Role: LEADER
I20260806 03:41:46.238034 19088 raft_consensus.cc:686] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 5 LEADER]: Becoming Leader. State: Replica: 491cdd4ce26949848b317caa81d5279d, State: Running, Role: LEADER
I20260806 03:41:46.239099 19088 consensus_queue.cc:237] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:46.238827 19087 consensus_queue.cc:260] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } }
I20260806 03:41:46.244046 19087 raft_consensus.cc:3037] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 4 FOLLOWER]: Advancing to term 5
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/ts_tablet_manager-itest.cc:914: Failure
Failed
Bad status: Remote error: Response had an error: code: NOT_THE_LEADER status { code: ILLEGAL_STATE message: "Not the leader. Local UUID: 30063023a8844495837679314257b563, Raft Consensus state: current_term: 5 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: \"30063023a8844495837679314257b563\" member_type: VOTER last_known_addr { host: \"127.15.252.126\" port: 42845 } } peers { permanent_uuid: \"2a5ff5ca39d348fa837104d428536561\" member_type: VOTER last_known_addr { host: \"127.15.252.125\" port: 34597 } } peers { permanent_uuid: \"c0591ed664d442fdaebaa2cd64c84b9f\" member_type: VOTER last_known_addr { host: \"127.15.252.124\" port: 39623 } } }" }
I20260806 03:41:46.250521 18899 raft_consensus.cc:504] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:46.255447 16369 tablet_server.cc:178] TabletServer@127.15.252.65:35231 shutting down...
I20260806 03:41:46.259148 18932 raft_consensus.cc:2379] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6eb7ccdcc221406296e866e74a37ed75 in current term 5: Already voted for candidate 491cdd4ce26949848b317caa81d5279d in this term.
I20260806 03:41:46.272064 18985 leader_election.cc:302] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6eb7ccdcc221406296e866e74a37ed75; no voters: 491cdd4ce26949848b317caa81d5279d, 4abe04524a7046018757fcdfd13b815d
I20260806 03:41:46.273586 19085 raft_consensus.cc:3037] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 4 FOLLOWER]: Advancing to term 5
I20260806 03:41:46.274967 18899 leader_election.cc:288] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 5 election: Requested vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 6eb7ccdcc221406296e866e74a37ed75 (127.15.252.67:35363)
W20260806 03:41:46.282505 18869 leader_election.cc:334] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260806 03:41:46.282946 18932 raft_consensus.cc:1226] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 5 FOLLOWER]: Rejecting Update request from peer 4abe04524a7046018757fcdfd13b815d for earlier term 4. Current term is 5. Ops: []
I20260806 03:41:46.284988 19087 consensus_queue.cc:1046] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 03:41:46.287377 19087 raft_consensus.cc:3032] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 4 LEADER]: Stepping down as leader of term 4
I20260806 03:41:46.288908 19087 raft_consensus.cc:729] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4abe04524a7046018757fcdfd13b815d, State: Running, Role: LEADER
I20260806 03:41:46.289048 19026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5390b9a07b5f4cd8b2330fd0974c8821" candidate_uuid: "491cdd4ce26949848b317caa81d5279d" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6eb7ccdcc221406296e866e74a37ed75"
I20260806 03:41:46.290109 19087 consensus_queue.cc:260] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:46.290983 18868 leader_election.cc:302] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d; no voters: 4abe04524a7046018757fcdfd13b815d, 6eb7ccdcc221406296e866e74a37ed75
I20260806 03:41:46.291105 19085 raft_consensus.cc:2726] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260806 03:41:46.291592 19087 raft_consensus.cc:3037] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 4 FOLLOWER]: Advancing to term 5
I20260806 03:41:46.292475 18899 raft_consensus.cc:2726] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260806 03:41:46.307304 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:46.308136 16369 tablet_replica.cc:330] stopping tablet replica
I20260806 03:41:46.308935 16369 raft_consensus.cc:2229] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 6 FOLLOWER]: Raft consensus shutting down.
I20260806 03:41:46.309432 16369 raft_consensus.cc:2258] T 6c23a83f61a74931bb2a5239d32e1cad P 4abe04524a7046018757fcdfd13b815d [term 6 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:46.312366 16369 tablet_replica.cc:330] stopping tablet replica
I20260806 03:41:46.313031 16369 raft_consensus.cc:2229] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 5 FOLLOWER]: Raft consensus shutting down.
I20260806 03:41:46.313481 16369 raft_consensus.cc:2258] T 4f6ce5ed631f413d907292ee85373f54 P 4abe04524a7046018757fcdfd13b815d [term 5 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:46.315685 16369 tablet_replica.cc:330] stopping tablet replica
I20260806 03:41:46.316234 16369 raft_consensus.cc:2229] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 5 FOLLOWER]: Raft consensus shutting down.
I20260806 03:41:46.316581 16369 raft_consensus.cc:2258] T 5390b9a07b5f4cd8b2330fd0974c8821 P 4abe04524a7046018757fcdfd13b815d [term 5 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:46.322573 19094 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 30063023a8844495837679314257b563)
I20260806 03:41:46.323024 19026 raft_consensus.cc:1261] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 5 FOLLOWER]: Refusing update from remote peer 491cdd4ce26949848b317caa81d5279d: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260806 03:41:46.323439 19094 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:46.327235 19094 leader_election.cc:288] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 30063023a8844495837679314257b563 (127.15.252.126:42845), 2a5ff5ca39d348fa837104d428536561 (127.15.252.125:34597)
I20260806 03:41:46.325982 18899 consensus_queue.cc:1035] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260806 03:41:46.328399 19033 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:41:46.328984 19033 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:46.333050 18701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30063023a8844495837679314257b563" candidate_term: 6 candidate_status { last_received { term: 4 index: 24 } } ignore_live_leader: false dest_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" is_pre_election: true
I20260806 03:41:46.334224 18701 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30063023a8844495837679314257b563 in term 4.
I20260806 03:41:46.334542 19033 leader_election.cc:288] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 2a5ff5ca39d348fa837104d428536561 (127.15.252.125:34597), c0591ed664d442fdaebaa2cd64c84b9f (127.15.252.124:39623)
I20260806 03:41:46.335793 18582 leader_election.cc:302] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30063023a8844495837679314257b563, c0591ed664d442fdaebaa2cd64c84b9f; no voters:
I20260806 03:41:46.336738 19033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260806 03:41:46.337095 19033 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:41:46.337430 19033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 5 FOLLOWER]: Advancing to term 6
I20260806 03:41:46.355762 19033 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:46.359346 18701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30063023a8844495837679314257b563" candidate_term: 6 candidate_status { last_received { term: 4 index: 24 } } ignore_live_leader: false dest_uuid: "c0591ed664d442fdaebaa2cd64c84b9f"
I20260806 03:41:46.359999 18701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 4 FOLLOWER]: Advancing to term 6
I20260806 03:41:46.367156 18701 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30063023a8844495837679314257b563 in term 6.
I20260806 03:41:46.368530 18582 leader_election.cc:302] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30063023a8844495837679314257b563, c0591ed664d442fdaebaa2cd64c84b9f; no voters:
I20260806 03:41:46.369133 19033 leader_election.cc:288] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [CANDIDATE]: Term 6 election: Requested vote from peers 2a5ff5ca39d348fa837104d428536561 (127.15.252.125:34597), c0591ed664d442fdaebaa2cd64c84b9f (127.15.252.124:39623)
I20260806 03:41:46.369514 18936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 6 FOLLOWER]: Leader election won for term 6
I20260806 03:41:46.369999 18936 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 6 LEADER]: Becoming Leader. State: Replica: 30063023a8844495837679314257b563, State: Running, Role: LEADER
I20260806 03:41:46.371119 18936 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 24, Committed index: 24, Last appended: 4.24, Last appended by leader: 24, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:46.373157 18606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" candidate_term: 5 candidate_status { last_received { term: 4 index: 24 } } ignore_live_leader: false dest_uuid: "30063023a8844495837679314257b563" is_pre_election: true
I20260806 03:41:46.374692 18936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30063023a8844495837679314257b563. Latest consensus state: current_term: 6 leader_uuid: "30063023a8844495837679314257b563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:46.375684 18936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:46.376524 19083 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:41:46.377070 19083 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:46.377298 18508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30063023a8844495837679314257b563" candidate_term: 6 candidate_status { last_received { term: 4 index: 24 } } ignore_live_leader: false dest_uuid: "2a5ff5ca39d348fa837104d428536561" is_pre_election: true
I20260806 03:41:46.378036 18508 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30063023a8844495837679314257b563 in term 5.
I20260806 03:41:46.378473 18507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" candidate_term: 5 candidate_status { last_received { term: 4 index: 24 } } ignore_live_leader: false dest_uuid: "2a5ff5ca39d348fa837104d428536561" is_pre_election: true
I20260806 03:41:46.379200 19083 leader_election.cc:288] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 30063023a8844495837679314257b563 (127.15.252.126:42845), c0591ed664d442fdaebaa2cd64c84b9f (127.15.252.124:39623)
I20260806 03:41:46.379513 18507 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c0591ed664d442fdaebaa2cd64c84b9f in current term 5: Already voted for candidate 2a5ff5ca39d348fa837104d428536561 in this term.
I20260806 03:41:46.379987 18509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "30063023a8844495837679314257b563" candidate_term: 6 candidate_status { last_received { term: 4 index: 24 } } ignore_live_leader: false dest_uuid: "2a5ff5ca39d348fa837104d428536561"
I20260806 03:41:46.380677 18509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 5 FOLLOWER]: Advancing to term 6
I20260806 03:41:46.380874 18676 leader_election.cc:302] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c0591ed664d442fdaebaa2cd64c84b9f; no voters: 2a5ff5ca39d348fa837104d428536561, 30063023a8844495837679314257b563
I20260806 03:41:46.382097 19094 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260806 03:41:46.383327 19100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:41:46.395112 18701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a5ff5ca39d348fa837104d428536561" candidate_term: 6 candidate_status { last_received { term: 4 index: 24 } } ignore_live_leader: false dest_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" is_pre_election: true
I20260806 03:41:46.399441 18701 raft_consensus.cc:2379] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 2a5ff5ca39d348fa837104d428536561 in current term 6: Already voted for candidate 30063023a8844495837679314257b563 in this term.
I20260806 03:41:46.400678 18509 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30063023a8844495837679314257b563 in term 6.
W20260806 03:41:46.404023 18869 consensus_peers.cc:487] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d -> Peer 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231): Couldn't send request to peer 4abe04524a7046018757fcdfd13b815d. Status: Network error: Client connection negotiation failed: client connection to 127.15.252.65:35231: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 03:41:46.404733 19085 raft_consensus.cc:482] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 491cdd4ce26949848b317caa81d5279d)
I20260806 03:41:46.405380 19085 raft_consensus.cc:504] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:46.409518 19085 leader_election.cc:288] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:46.409732 18932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d" is_pre_election: true
I20260806 03:41:46.410467 18932 raft_consensus.cc:2454] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6eb7ccdcc221406296e866e74a37ed75 in term 6.
I20260806 03:41:46.410919 19100 catalog_manager.cc:669] Loaded metadata for table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe]
I20260806 03:41:46.412278 18985 leader_election.cc:302] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d, 6eb7ccdcc221406296e866e74a37ed75; no voters:
I20260806 03:41:46.413125 19085 raft_consensus.cc:2781] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260806 03:41:46.413506 19085 raft_consensus.cc:482] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 491cdd4ce26949848b317caa81d5279d)
I20260806 03:41:46.413825 19085 raft_consensus.cc:3037] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 5 FOLLOWER]: Advancing to term 6
I20260806 03:41:46.414794 16369 tablet_server.cc:194] TabletServer@127.15.252.65:35231 shutdown complete.
W20260806 03:41:46.418866 18987 leader_election.cc:334] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231): Network error: Client connection negotiation failed: client connection to 127.15.252.65:35231: connect: Connection refused (error 111)
I20260806 03:41:46.420884 19085 raft_consensus.cc:504] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:46.423051 19085 leader_election.cc:288] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 6 election: Requested vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
I20260806 03:41:46.424544 18932 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c23a83f61a74931bb2a5239d32e1cad" candidate_uuid: "6eb7ccdcc221406296e866e74a37ed75" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "491cdd4ce26949848b317caa81d5279d"
W20260806 03:41:46.428208 18987 leader_election.cc:334] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231): Network error: Client connection negotiation failed: client connection to 127.15.252.65:35231: connect: Connection refused (error 111)
I20260806 03:41:46.429647 19100 tablet_loader.cc:96] loaded metadata for tablet 4f6ce5ed631f413d907292ee85373f54 (table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe])
I20260806 03:41:46.430958 19100 tablet_loader.cc:96] loaded metadata for tablet 5390b9a07b5f4cd8b2330fd0974c8821 (table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe])
I20260806 03:41:46.432062 19100 tablet_loader.cc:96] loaded metadata for tablet 6c23a83f61a74931bb2a5239d32e1cad (table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe])
I20260806 03:41:46.432333 18932 raft_consensus.cc:2454] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6eb7ccdcc221406296e866e74a37ed75 in term 6.
I20260806 03:41:46.433421 19100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:41:46.433559 18606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a5ff5ca39d348fa837104d428536561" candidate_term: 6 candidate_status { last_received { term: 4 index: 24 } } ignore_live_leader: false dest_uuid: "30063023a8844495837679314257b563" is_pre_election: true
I20260806 03:41:46.433672 18985 leader_election.cc:302] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 491cdd4ce26949848b317caa81d5279d, 6eb7ccdcc221406296e866e74a37ed75; no voters: 4abe04524a7046018757fcdfd13b815d
I20260806 03:41:46.434625 19037 raft_consensus.cc:2781] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 6 FOLLOWER]: Leader election won for term 6
I20260806 03:41:46.435184 19037 raft_consensus.cc:686] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 6 LEADER]: Becoming Leader. State: Replica: 6eb7ccdcc221406296e866e74a37ed75, State: Running, Role: LEADER
I20260806 03:41:46.435343 18481 leader_election.cc:302] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a5ff5ca39d348fa837104d428536561; no voters: 30063023a8844495837679314257b563, c0591ed664d442fdaebaa2cd64c84b9f
I20260806 03:41:46.436537 19083 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260806 03:41:46.436056 19037 consensus_queue.cc:237] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:46.437049 16369 tablet_server.cc:178] TabletServer@127.15.252.66:43729 shutting down...
I20260806 03:41:46.440690 19100 catalog_manager.cc:1259] Loaded cluster ID: 8225191a001148569992571dec22b329
I20260806 03:41:46.441124 19100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:41:46.452778 19100 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:41:46.457640 19100 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563: Loaded TSK: 0
I20260806 03:41:46.459247 19100 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:41:46.462153 18701 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 6 FOLLOWER]: Refusing update from remote peer 30063023a8844495837679314257b563: Log matching property violated. Preceding OpId in replica: term: 4 index: 24. Preceding OpId from leader: term: 6 index: 25. (index mismatch)
I20260806 03:41:46.464527 18936 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.001s
I20260806 03:41:46.474982 19094 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30063023a8844495837679314257b563. Latest consensus state: current_term: 6 leader_uuid: "30063023a8844495837679314257b563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:46.476050 19094 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:46.480504 18936 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "30063023a8844495837679314257b563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:46.481330 18509 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 6 FOLLOWER]: Refusing update from remote peer 30063023a8844495837679314257b563: Log matching property violated. Preceding OpId in replica: term: 4 index: 24. Preceding OpId from leader: term: 6 index: 25. (index mismatch)
I20260806 03:41:46.481432 18936 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:46.482983 18936 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.001s
I20260806 03:41:46.483386 19094 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "30063023a8844495837679314257b563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:46.484297 19094 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:46.493511 19106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 6 leader_uuid: "30063023a8844495837679314257b563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:46.494447 19106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:46.494117 19083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30063023a8844495837679314257b563. Latest consensus state: current_term: 6 leader_uuid: "30063023a8844495837679314257b563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:46.495036 19083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:46.496206 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:46.496968 16369 tablet_replica.cc:330] stopping tablet replica
I20260806 03:41:46.497712 16369 raft_consensus.cc:2229] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 6 FOLLOWER]: Raft consensus shutting down.
I20260806 03:41:46.498080 16369 raft_consensus.cc:2258] T 6c23a83f61a74931bb2a5239d32e1cad P 491cdd4ce26949848b317caa81d5279d [term 6 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:46.500317 16369 tablet_replica.cc:330] stopping tablet replica
I20260806 03:41:46.500756 16369 raft_consensus.cc:2229] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 5 LEADER]: Raft consensus shutting down.
I20260806 03:41:46.501467 16369 raft_consensus.cc:2258] T 4f6ce5ed631f413d907292ee85373f54 P 491cdd4ce26949848b317caa81d5279d [term 5 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:46.503901 16369 tablet_replica.cc:330] stopping tablet replica
I20260806 03:41:46.504341 16369 raft_consensus.cc:2229] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 5 FOLLOWER]: Raft consensus shutting down.
I20260806 03:41:46.504714 16369 raft_consensus.cc:2258] T 5390b9a07b5f4cd8b2330fd0974c8821 P 491cdd4ce26949848b317caa81d5279d [term 5 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:46.519371 16369 tablet_server.cc:194] TabletServer@127.15.252.66:43729 shutdown complete.
I20260806 03:41:46.531281 16369 tablet_server.cc:178] TabletServer@127.15.252.67:35363 shutting down...
W20260806 03:41:46.546602 18985 consensus_peers.cc:487] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 -> Peer 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729): Couldn't send request to peer 491cdd4ce26949848b317caa81d5279d. Status: Network error: Client connection negotiation failed: client connection to 127.15.252.66:43729: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 03:41:46.559173 19037 raft_consensus.cc:482] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:41:46.559674 19037 raft_consensus.cc:504] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4abe04524a7046018757fcdfd13b815d" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35231 } } peers { permanent_uuid: "491cdd4ce26949848b317caa81d5279d" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 43729 } } peers { permanent_uuid: "6eb7ccdcc221406296e866e74a37ed75" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35363 } }
I20260806 03:41:46.561151 19037 leader_election.cc:288] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231), 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729)
W20260806 03:41:46.563611 18987 consensus_peers.cc:487] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 -> Peer 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231): Couldn't send request to peer 4abe04524a7046018757fcdfd13b815d. Status: Network error: Client connection negotiation failed: client connection to 127.15.252.65:35231: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 03:41:46.564824 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:46.565475 16369 tablet_replica.cc:330] stopping tablet replica
I20260806 03:41:46.566095 16369 raft_consensus.cc:2229] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 6 LEADER]: Raft consensus shutting down.
I20260806 03:41:46.567020 16369 pending_rounds.cc:62] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75: Trying to abort 1 pending ops.
W20260806 03:41:46.567322 18985 leader_election.cc:334] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 491cdd4ce26949848b317caa81d5279d (127.15.252.66:43729): Network error: Client connection negotiation failed: client connection to 127.15.252.66:43729: connect: Connection refused (error 111)
I20260806 03:41:46.567277 16369 pending_rounds.cc:68] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75: Aborting op as it isn't in flight: id { term: 6 index: 8 } timestamp: 7315405645574344704 op_type: NO_OP noop_request { }
I20260806 03:41:46.567924 16369 raft_consensus.cc:2866] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 6 LEADER]: NO_OP replication failed: Aborted: Op aborted
W20260806 03:41:46.567905 18987 leader_election.cc:334] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 4abe04524a7046018757fcdfd13b815d (127.15.252.65:35231): Network error: Client connection negotiation failed: client connection to 127.15.252.65:35231: connect: Connection refused (error 111)
I20260806 03:41:46.568486 16369 raft_consensus.cc:2258] T 6c23a83f61a74931bb2a5239d32e1cad P 6eb7ccdcc221406296e866e74a37ed75 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:46.568529 18987 leader_election.cc:302] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6eb7ccdcc221406296e866e74a37ed75; no voters: 491cdd4ce26949848b317caa81d5279d, 4abe04524a7046018757fcdfd13b815d
I20260806 03:41:46.570086 19037 raft_consensus.cc:2726] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260806 03:41:46.571499 16369 tablet_replica.cc:330] stopping tablet replica
I20260806 03:41:46.572054 16369 raft_consensus.cc:2229] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260806 03:41:46.572466 16369 raft_consensus.cc:2258] T 4f6ce5ed631f413d907292ee85373f54 P 6eb7ccdcc221406296e866e74a37ed75 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:46.575268 16369 tablet_replica.cc:330] stopping tablet replica
I20260806 03:41:46.575755 16369 raft_consensus.cc:2229] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260806 03:41:46.576134 16369 raft_consensus.cc:2258] T 5390b9a07b5f4cd8b2330fd0974c8821 P 6eb7ccdcc221406296e866e74a37ed75 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:46.579924 19104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 6 leader_uuid: "30063023a8844495837679314257b563" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:46.580794 19104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:46.592926 16369 tablet_server.cc:194] TabletServer@127.15.252.67:35363 shutdown complete.
I20260806 03:41:46.605007 16369 master.cc:537] Master@127.15.252.126:42845 shutting down...
I20260806 03:41:46.625299 16369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 6 LEADER]: Raft consensus shutting down.
I20260806 03:41:46.626103 16369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 30063023a8844495837679314257b563 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:46.626551 16369 tablet_replica.cc:330] stopping tablet replica
I20260806 03:41:46.782378 19083 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 30063023a8844495837679314257b563)
I20260806 03:41:46.783013 19083 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:46.785087 19083 leader_election.cc:288] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 30063023a8844495837679314257b563 (127.15.252.126:42845), c0591ed664d442fdaebaa2cd64c84b9f (127.15.252.124:39623)
I20260806 03:41:46.786792 18701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2a5ff5ca39d348fa837104d428536561" candidate_term: 7 candidate_status { last_received { term: 6 index: 25 } } ignore_live_leader: false dest_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" is_pre_election: true
W20260806 03:41:46.790184 18481 leader_election.cc:334] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 30063023a8844495837679314257b563 (127.15.252.126:42845): Network error: Client connection negotiation failed: client connection to 127.15.252.126:42845: connect: Connection refused (error 111)
I20260806 03:41:46.790565 18481 leader_election.cc:302] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2a5ff5ca39d348fa837104d428536561; no voters: 30063023a8844495837679314257b563, c0591ed664d442fdaebaa2cd64c84b9f
I20260806 03:41:46.791252 19083 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260806 03:41:46.799172 19094 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 30063023a8844495837679314257b563)
I20260806 03:41:46.799773 19094 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:46.801482 19094 leader_election.cc:288] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 30063023a8844495837679314257b563 (127.15.252.126:42845), 2a5ff5ca39d348fa837104d428536561 (127.15.252.125:34597)
I20260806 03:41:46.802793 18509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" candidate_term: 7 candidate_status { last_received { term: 6 index: 25 } } ignore_live_leader: false dest_uuid: "2a5ff5ca39d348fa837104d428536561" is_pre_election: true
I20260806 03:41:46.803560 18509 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0591ed664d442fdaebaa2cd64c84b9f in term 6.
I20260806 03:41:46.805092 18676 leader_election.cc:302] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a5ff5ca39d348fa837104d428536561, c0591ed664d442fdaebaa2cd64c84b9f; no voters:
I20260806 03:41:46.805992 19094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260806 03:41:46.806180 18674 leader_election.cc:334] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 30063023a8844495837679314257b563 (127.15.252.126:42845): Network error: Client connection negotiation failed: client connection to 127.15.252.126:42845: connect: Connection refused (error 111)
I20260806 03:41:46.806363 19094 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 30063023a8844495837679314257b563)
I20260806 03:41:46.806804 19094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 6 FOLLOWER]: Advancing to term 7
I20260806 03:41:46.811843 19094 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:46.813405 19094 leader_election.cc:288] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [CANDIDATE]: Term 7 election: Requested vote from peers 30063023a8844495837679314257b563 (127.15.252.126:42845), 2a5ff5ca39d348fa837104d428536561 (127.15.252.125:34597)
I20260806 03:41:46.814604 18509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" candidate_term: 7 candidate_status { last_received { term: 6 index: 25 } } ignore_live_leader: false dest_uuid: "2a5ff5ca39d348fa837104d428536561"
I20260806 03:41:46.815330 18509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 6 FOLLOWER]: Advancing to term 7
W20260806 03:41:46.816777 18674 leader_election.cc:334] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 30063023a8844495837679314257b563 (127.15.252.126:42845): Network error: Client connection negotiation failed: client connection to 127.15.252.126:42845: connect: Connection refused (error 111)
I20260806 03:41:46.821077 18509 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0591ed664d442fdaebaa2cd64c84b9f in term 7.
I20260806 03:41:46.822597 18676 leader_election.cc:302] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2a5ff5ca39d348fa837104d428536561, c0591ed664d442fdaebaa2cd64c84b9f; no voters: 30063023a8844495837679314257b563
I20260806 03:41:46.823454 19094 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 7 FOLLOWER]: Leader election won for term 7
I20260806 03:41:46.824625 19094 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 7 LEADER]: Becoming Leader. State: Replica: c0591ed664d442fdaebaa2cd64c84b9f, State: Running, Role: LEADER
I20260806 03:41:46.825361 19094 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 6.25, Last appended by leader: 25, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } }
I20260806 03:41:46.828192 19110 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0591ed664d442fdaebaa2cd64c84b9f. Latest consensus state: current_term: 7 leader_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:46.828929 19110 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:46.830421 19111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:41:46.835258 19111 catalog_manager.cc:669] Loaded metadata for table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe]
I20260806 03:41:46.839579 19111 tablet_loader.cc:96] loaded metadata for tablet 4f6ce5ed631f413d907292ee85373f54 (table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe])
I20260806 03:41:46.840623 19111 tablet_loader.cc:96] loaded metadata for tablet 5390b9a07b5f4cd8b2330fd0974c8821 (table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe])
I20260806 03:41:46.841601 19111 tablet_loader.cc:96] loaded metadata for tablet 6c23a83f61a74931bb2a5239d32e1cad (table NewTableName [id=325a9a6c58fd4d258c4388b45595aefe])
I20260806 03:41:46.842527 19111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:41:46.846005 19111 catalog_manager.cc:1259] Loaded cluster ID: 8225191a001148569992571dec22b329
I20260806 03:41:46.846264 19111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:41:46.850323 19111 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:41:46.853808 19111 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f: Loaded TSK: 0
I20260806 03:41:46.855087 19111 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 03:41:46.928084 18628 debug-util.cc:398] Leaking SignalData structure 0x7b080018ea20 after lost signal to thread 16372
W20260806 03:41:46.929330 18674 consensus_peers.cc:487] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f -> Peer 30063023a8844495837679314257b563 (127.15.252.126:42845): Couldn't send request to peer 30063023a8844495837679314257b563. Status: Network error: Client connection negotiation failed: client connection to 127.15.252.126:42845: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 03:41:46.954679 18509 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 7 FOLLOWER]: Refusing update from remote peer c0591ed664d442fdaebaa2cd64c84b9f: Log matching property violated. Preceding OpId in replica: term: 6 index: 25. Preceding OpId from leader: term: 7 index: 26. (index mismatch)
I20260806 03:41:46.956321 19110 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20260806 03:41:46.959971 19083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0591ed664d442fdaebaa2cd64c84b9f. Latest consensus state: current_term: 7 leader_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:46.960995 19083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:41:46.963537 19112 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 7 leader_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:46.964296 19112 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:46.965801 19083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 7 leader_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30063023a8844495837679314257b563" member_type: VOTER last_known_addr { host: "127.15.252.126" port: 42845 } } peers { permanent_uuid: "2a5ff5ca39d348fa837104d428536561" member_type: VOTER last_known_addr { host: "127.15.252.125" port: 34597 } } peers { permanent_uuid: "c0591ed664d442fdaebaa2cd64c84b9f" member_type: VOTER last_known_addr { host: "127.15.252.124" port: 39623 } } }
I20260806 03:41:46.966849 19083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 03:41:47.049470 18674 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.252.126:42845: connect: Connection refused (error 111) [suppressed 67 similar messages]
W20260806 03:41:47.457753 18674 consensus_peers.cc:487] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f -> Peer 30063023a8844495837679314257b563 (127.15.252.126:42845): Couldn't send request to peer 30063023a8844495837679314257b563. Status: Network error: Client connection negotiation failed: client connection to 127.15.252.126:42845: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260806 03:41:47.644840 16369 thread.cc:536] Waited for 1000ms trying to join with diag-logger (tid 18628)
W20260806 03:41:47.933241 18674 consensus_peers.cc:487] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f -> Peer 30063023a8844495837679314257b563 (127.15.252.126:42845): Couldn't send request to peer 30063023a8844495837679314257b563. Status: Network error: Client connection negotiation failed: client connection to 127.15.252.126:42845: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260806 03:41:48.435914 18674 consensus_peers.cc:487] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f -> Peer 30063023a8844495837679314257b563 (127.15.252.126:42845): Couldn't send request to peer 30063023a8844495837679314257b563. Status: Network error: Client connection negotiation failed: client connection to 127.15.252.126:42845: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260806 03:41:48.645314 16369 thread.cc:536] Waited for 2000ms trying to join with diag-logger (tid 18628)
W20260806 03:41:48.966992 18674 consensus_peers.cc:487] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f -> Peer 30063023a8844495837679314257b563 (127.15.252.126:42845): Couldn't send request to peer 30063023a8844495837679314257b563. Status: Network error: Client connection negotiation failed: client connection to 127.15.252.126:42845: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260806 03:41:49.501250 18674 consensus_peers.cc:487] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f -> Peer 30063023a8844495837679314257b563 (127.15.252.126:42845): Couldn't send request to peer 30063023a8844495837679314257b563. Status: Network error: Client connection negotiation failed: client connection to 127.15.252.126:42845: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260806 03:41:49.646130 16369 thread.cc:536] Waited for 3000ms trying to join with diag-logger (tid 18628)
W20260806 03:41:50.016152 18674 consensus_peers.cc:487] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f -> Peer 30063023a8844495837679314257b563 (127.15.252.126:42845): Couldn't send request to peer 30063023a8844495837679314257b563. Status: Network error: Client connection negotiation failed: client connection to 127.15.252.126:42845: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260806 03:41:50.571282 18674 consensus_peers.cc:487] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f -> Peer 30063023a8844495837679314257b563 (127.15.252.126:42845): Couldn't send request to peer 30063023a8844495837679314257b563. Status: Network error: Client connection negotiation failed: client connection to 127.15.252.126:42845: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260806 03:41:50.647238 16369 thread.cc:536] Waited for 4000ms trying to join with diag-logger (tid 18628)
I20260806 03:41:51.000521 16369 master.cc:552] Master@127.15.252.126:42845 shutdown complete.
I20260806 03:41:51.026935 16369 master.cc:537] Master@127.15.252.125:34597 shutting down...
I20260806 03:41:51.055030 16369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 7 FOLLOWER]: Raft consensus shutting down.
I20260806 03:41:51.055632 16369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2a5ff5ca39d348fa837104d428536561 [term 7 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:51.055948 16369 tablet_replica.cc:330] stopping tablet replica
W20260806 03:41:51.075660 16369 rolling_log.cc:182] Unable to compress old log file: Not found: Unable to open input file to compress: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0/ts_tablet_manager-itest.dist-test-slave-csg5.slave.diagnostics.20260806-034141.0.16369: No such file or directory (error 2)
I20260806 03:41:51.076510 16369 master.cc:552] Master@127.15.252.125:34597 shutdown complete.
I20260806 03:41:51.094429 16369 master.cc:537] Master@127.15.252.124:39623 shutting down...
I20260806 03:41:51.120854 16369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 7 LEADER]: Raft consensus shutting down.
I20260806 03:41:51.122318 16369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c0591ed664d442fdaebaa2cd64c84b9f [term 7 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:51.122833 16369 tablet_replica.cc:330] stopping tablet replica
I20260806 03:41:51.144779 16369 master.cc:552] Master@127.15.252.124:39623 shutdown complete.
I20260806 03:41:51.184887 16369 test_util.cc:175] -----------------------------------------------
I20260806 03:41:51.185189 16369 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestTableStats.1785987678583700-16369-0
[ FAILED ] TsTabletManagerITest.TestTableStats (14701 ms)
[ RUN ] TsTabletManagerITest.TestDeleteTableDuringTabletCopy
I20260806 03:41:51.193681 16369 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.252.126:43741
I20260806 03:41:51.195086 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:51.202704 19119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:51.203855 19120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:51.204568 19122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:51.205617 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:51.206516 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:51.206774 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:51.206897 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987711206889 us; error 0 us; skew 500 ppm
I20260806 03:41:51.207407 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:51.210212 16369 webserver.cc:460] Webserver started at http://127.15.252.126:37483/ using document root <none> and password file <none>
I20260806 03:41:51.210798 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:51.210968 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:51.211189 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:51.212656 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/master-0-root/instance:
uuid: "613e8a2f330d4f4dacbf5bb7e1a506f2"
format_stamp: "Formatted at 2026-08-06 03:41:51 on dist-test-slave-csg5"
I20260806 03:41:51.217813 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 03:41:51.222311 19127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:51.223363 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 03:41:51.223716 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/master-0-root
uuid: "613e8a2f330d4f4dacbf5bb7e1a506f2"
format_stamp: "Formatted at 2026-08-06 03:41:51 on dist-test-slave-csg5"
I20260806 03:41:51.224078 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:51.256152 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:51.257704 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:51.303994 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.126:43741
I20260806 03:41:51.304113 19180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.126:43741 every 8 connection(s)
I20260806 03:41:51.308378 19181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:51.319751 19181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2: Bootstrap starting.
I20260806 03:41:51.324793 19181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:51.329640 19181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2: No bootstrap required, opened a new log
I20260806 03:41:51.331904 19181 raft_consensus.cc:348] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "613e8a2f330d4f4dacbf5bb7e1a506f2" member_type: VOTER }
I20260806 03:41:51.332331 19181 raft_consensus.cc:374] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:51.332525 19181 raft_consensus.cc:729] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 613e8a2f330d4f4dacbf5bb7e1a506f2, State: Initialized, Role: FOLLOWER
I20260806 03:41:51.333032 19181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "613e8a2f330d4f4dacbf5bb7e1a506f2" member_type: VOTER }
I20260806 03:41:51.333477 19181 raft_consensus.cc:388] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:41:51.333689 19181 raft_consensus.cc:482] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:41:51.333922 19181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:51.338421 19181 raft_consensus.cc:504] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "613e8a2f330d4f4dacbf5bb7e1a506f2" member_type: VOTER }
I20260806 03:41:51.339028 19181 leader_election.cc:302] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 613e8a2f330d4f4dacbf5bb7e1a506f2; no voters:
I20260806 03:41:51.340372 19181 leader_election.cc:288] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:41:51.340857 19184 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:51.342468 19184 raft_consensus.cc:686] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2 [term 1 LEADER]: Becoming Leader. State: Replica: 613e8a2f330d4f4dacbf5bb7e1a506f2, State: Running, Role: LEADER
I20260806 03:41:51.343281 19184 consensus_queue.cc:237] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "613e8a2f330d4f4dacbf5bb7e1a506f2" member_type: VOTER }
I20260806 03:41:51.343510 19181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:41:51.346876 19185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "613e8a2f330d4f4dacbf5bb7e1a506f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "613e8a2f330d4f4dacbf5bb7e1a506f2" member_type: VOTER } }
I20260806 03:41:51.347570 19185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2 [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:51.347709 19186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 613e8a2f330d4f4dacbf5bb7e1a506f2. Latest consensus state: current_term: 1 leader_uuid: "613e8a2f330d4f4dacbf5bb7e1a506f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "613e8a2f330d4f4dacbf5bb7e1a506f2" member_type: VOTER } }
I20260806 03:41:51.348724 19186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2 [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:51.354439 19190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:41:51.361140 19190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:41:51.362324 16369 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260806 03:41:51.371649 19190 catalog_manager.cc:1347] Generated new cluster ID: ec94bbb4a5fd4958995186c2cbc677ed
I20260806 03:41:51.372012 19190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:41:51.387264 19190 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:41:51.389254 19190 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:41:51.408857 19190 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2: Generated new TSK 0
I20260806 03:41:51.409616 19190 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:41:51.429160 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:51.436257 19202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:51.437844 19203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:51.438818 19205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:51.439265 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:51.441227 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:51.441458 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:51.441634 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987711441620 us; error 0 us; skew 500 ppm
I20260806 03:41:51.442226 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:51.445248 16369 webserver.cc:460] Webserver started at http://127.15.252.65:44577/ using document root <none> and password file <none>
I20260806 03:41:51.445794 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:51.446007 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:51.446281 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:51.447580 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/ts-0-root/instance:
uuid: "f5f38a0561514bd7880a57a0cd58e796"
format_stamp: "Formatted at 2026-08-06 03:41:51 on dist-test-slave-csg5"
I20260806 03:41:51.452613 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260806 03:41:51.457067 19211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:51.458110 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 03:41:51.458483 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/ts-0-root
uuid: "f5f38a0561514bd7880a57a0cd58e796"
format_stamp: "Formatted at 2026-08-06 03:41:51 on dist-test-slave-csg5"
I20260806 03:41:51.458910 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:51.472163 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:51.473362 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:51.475160 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:51.477947 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:51.478200 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:51.478464 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:51.478641 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:51.527184 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.65:38895
I20260806 03:41:51.527344 19275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.65:38895 every 8 connection(s)
I20260806 03:41:51.532699 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:51.541795 19281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:51.542061 19280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:51.543176 19283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:51.544766 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:51.545872 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:51.546092 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:51.546284 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987711546273 us; error 0 us; skew 500 ppm
I20260806 03:41:51.547017 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:51.549019 19276 heartbeater.cc:347] Connected to a master server at 127.15.252.126:43741
I20260806 03:41:51.549430 19276 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:51.550076 16369 webserver.cc:460] Webserver started at http://127.15.252.66:36409/ using document root <none> and password file <none>
I20260806 03:41:51.550351 19276 heartbeater.cc:511] Master 127.15.252.126:43741 requested a full tablet report, sending...
I20260806 03:41:51.550621 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:51.550967 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:51.551244 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:51.552637 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/ts-1-root/instance:
uuid: "4e3398149bf24815ae2e76918bd0b27c"
format_stamp: "Formatted at 2026-08-06 03:41:51 on dist-test-slave-csg5"
I20260806 03:41:51.553373 19145 ts_manager.cc:194] Registered new tserver with Master: f5f38a0561514bd7880a57a0cd58e796 (127.15.252.65:38895)
I20260806 03:41:51.555815 19145 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51354
I20260806 03:41:51.557982 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260806 03:41:51.562294 19288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:51.563239 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 03:41:51.563556 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/ts-1-root
uuid: "4e3398149bf24815ae2e76918bd0b27c"
format_stamp: "Formatted at 2026-08-06 03:41:51 on dist-test-slave-csg5"
I20260806 03:41:51.563860 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:51.592051 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:51.593303 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:51.594985 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:51.597337 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:51.597565 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:51.597812 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:51.597983 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:51.642158 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.66:38021
I20260806 03:41:51.642275 19354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.66:38021 every 8 connection(s)
I20260806 03:41:51.647425 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:51.654510 19358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:51.655826 19359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:51.659857 19361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:51.660221 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:51.661458 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:51.661736 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:51.661957 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987711661938 us; error 0 us; skew 500 ppm
I20260806 03:41:51.662549 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:51.662961 19355 heartbeater.cc:347] Connected to a master server at 127.15.252.126:43741
I20260806 03:41:51.663405 19355 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:51.664273 19355 heartbeater.cc:511] Master 127.15.252.126:43741 requested a full tablet report, sending...
I20260806 03:41:51.666121 16369 webserver.cc:460] Webserver started at http://127.15.252.67:40537/ using document root <none> and password file <none>
I20260806 03:41:51.666806 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:51.666791 19145 ts_manager.cc:194] Registered new tserver with Master: 4e3398149bf24815ae2e76918bd0b27c (127.15.252.66:38021)
I20260806 03:41:51.667259 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:51.667629 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:51.668787 19145 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51370
I20260806 03:41:51.669000 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/ts-2-root/instance:
uuid: "c2f74e482ee44565ae0e82b205783619"
format_stamp: "Formatted at 2026-08-06 03:41:51 on dist-test-slave-csg5"
I20260806 03:41:51.675315 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260806 03:41:51.679715 19366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:51.680636 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 03:41:51.680998 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/ts-2-root
uuid: "c2f74e482ee44565ae0e82b205783619"
format_stamp: "Formatted at 2026-08-06 03:41:51 on dist-test-slave-csg5"
I20260806 03:41:51.681326 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:51.695190 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:51.696593 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:51.698263 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:51.701191 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:51.701574 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:51.701856 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:51.702049 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:51.751209 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.67:43307
I20260806 03:41:51.751313 19431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.67:43307 every 8 connection(s)
I20260806 03:41:51.771016 19432 heartbeater.cc:347] Connected to a master server at 127.15.252.126:43741
I20260806 03:41:51.771477 19432 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:51.772238 19432 heartbeater.cc:511] Master 127.15.252.126:43741 requested a full tablet report, sending...
I20260806 03:41:51.774158 19145 ts_manager.cc:194] Registered new tserver with Master: c2f74e482ee44565ae0e82b205783619 (127.15.252.67:43307)
I20260806 03:41:51.775259 16369 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.020117186s
I20260806 03:41:51.776001 19145 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51380
I20260806 03:41:51.818506 19145 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51392:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
}
}
W20260806 03:41:51.820174 19145 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:41:51.868436 19318 tablet_service.cc:1467] Processing CreateTablet for tablet dec6183f982c4fe9adfae4666cc68eee (DEFAULT_TABLE table=test-table [id=3955dc6c9edb4d5893cc40be0d696781]), partition=
I20260806 03:41:51.869666 19318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dec6183f982c4fe9adfae4666cc68eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:51.873265 19395 tablet_service.cc:1467] Processing CreateTablet for tablet dec6183f982c4fe9adfae4666cc68eee (DEFAULT_TABLE table=test-table [id=3955dc6c9edb4d5893cc40be0d696781]), partition=
I20260806 03:41:51.874430 19395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dec6183f982c4fe9adfae4666cc68eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:51.880796 19241 tablet_service.cc:1467] Processing CreateTablet for tablet dec6183f982c4fe9adfae4666cc68eee (DEFAULT_TABLE table=test-table [id=3955dc6c9edb4d5893cc40be0d696781]), partition=
I20260806 03:41:51.882043 19241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dec6183f982c4fe9adfae4666cc68eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:51.887852 19450 tablet_bootstrap.cc:492] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c: Bootstrap starting.
I20260806 03:41:51.890581 19451 tablet_bootstrap.cc:492] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: Bootstrap starting.
I20260806 03:41:51.896190 19450 tablet_bootstrap.cc:654] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:51.897313 19451 tablet_bootstrap.cc:654] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:51.901934 19452 tablet_bootstrap.cc:492] T dec6183f982c4fe9adfae4666cc68eee P f5f38a0561514bd7880a57a0cd58e796: Bootstrap starting.
I20260806 03:41:51.907043 19450 tablet_bootstrap.cc:492] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c: No bootstrap required, opened a new log
I20260806 03:41:51.907083 19451 tablet_bootstrap.cc:492] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: No bootstrap required, opened a new log
I20260806 03:41:51.907641 19450 ts_tablet_manager.cc:1397] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c: Time spent bootstrapping tablet: real 0.020s user 0.014s sys 0.000s
I20260806 03:41:51.907722 19451 ts_tablet_manager.cc:1397] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.013s
I20260806 03:41:51.910197 19452 tablet_bootstrap.cc:654] T dec6183f982c4fe9adfae4666cc68eee P f5f38a0561514bd7880a57a0cd58e796: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:51.910207 19450 raft_consensus.cc:348] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3398149bf24815ae2e76918bd0b27c" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 38021 } } peers { permanent_uuid: "c2f74e482ee44565ae0e82b205783619" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43307 } } peers { permanent_uuid: "f5f38a0561514bd7880a57a0cd58e796" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 38895 } }
I20260806 03:41:51.911486 19450 raft_consensus.cc:374] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:51.911233 19451 raft_consensus.cc:348] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3398149bf24815ae2e76918bd0b27c" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 38021 } } peers { permanent_uuid: "c2f74e482ee44565ae0e82b205783619" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43307 } } peers { permanent_uuid: "f5f38a0561514bd7880a57a0cd58e796" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 38895 } }
I20260806 03:41:51.911921 19450 raft_consensus.cc:729] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e3398149bf24815ae2e76918bd0b27c, State: Initialized, Role: FOLLOWER
I20260806 03:41:51.912247 19451 raft_consensus.cc:374] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:51.912662 19451 raft_consensus.cc:729] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2f74e482ee44565ae0e82b205783619, State: Initialized, Role: FOLLOWER
I20260806 03:41:51.912811 19450 consensus_queue.cc:260] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3398149bf24815ae2e76918bd0b27c" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 38021 } } peers { permanent_uuid: "c2f74e482ee44565ae0e82b205783619" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43307 } } peers { permanent_uuid: "f5f38a0561514bd7880a57a0cd58e796" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 38895 } }
I20260806 03:41:51.913652 19451 consensus_queue.cc:260] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3398149bf24815ae2e76918bd0b27c" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 38021 } } peers { permanent_uuid: "c2f74e482ee44565ae0e82b205783619" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43307 } } peers { permanent_uuid: "f5f38a0561514bd7880a57a0cd58e796" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 38895 } }
I20260806 03:41:51.916674 19355 heartbeater.cc:503] Master 127.15.252.126:43741 was elected leader, sending a full tablet report...
I20260806 03:41:51.917151 19450 ts_tablet_manager.cc:1428] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
W20260806 03:41:51.917466 19450 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260806 03:41:51.920436 19432 heartbeater.cc:503] Master 127.15.252.126:43741 was elected leader, sending a full tablet report...
I20260806 03:41:51.920825 19452 tablet_bootstrap.cc:492] T dec6183f982c4fe9adfae4666cc68eee P f5f38a0561514bd7880a57a0cd58e796: No bootstrap required, opened a new log
I20260806 03:41:51.921057 19451 ts_tablet_manager.cc:1428] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: Time spent starting tablet: real 0.013s user 0.004s sys 0.009s
W20260806 03:41:51.921378 19451 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260806 03:41:51.921447 19452 ts_tablet_manager.cc:1397] T dec6183f982c4fe9adfae4666cc68eee P f5f38a0561514bd7880a57a0cd58e796: Time spent bootstrapping tablet: real 0.020s user 0.018s sys 0.000s
I20260806 03:41:51.924868 19452 raft_consensus.cc:348] T dec6183f982c4fe9adfae4666cc68eee P f5f38a0561514bd7880a57a0cd58e796 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3398149bf24815ae2e76918bd0b27c" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 38021 } } peers { permanent_uuid: "c2f74e482ee44565ae0e82b205783619" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43307 } } peers { permanent_uuid: "f5f38a0561514bd7880a57a0cd58e796" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 38895 } }
I20260806 03:41:51.925782 19452 raft_consensus.cc:374] T dec6183f982c4fe9adfae4666cc68eee P f5f38a0561514bd7880a57a0cd58e796 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:51.926120 19452 raft_consensus.cc:729] T dec6183f982c4fe9adfae4666cc68eee P f5f38a0561514bd7880a57a0cd58e796 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5f38a0561514bd7880a57a0cd58e796, State: Initialized, Role: FOLLOWER
I20260806 03:41:51.926957 19452 consensus_queue.cc:260] T dec6183f982c4fe9adfae4666cc68eee P f5f38a0561514bd7880a57a0cd58e796 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3398149bf24815ae2e76918bd0b27c" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 38021 } } peers { permanent_uuid: "c2f74e482ee44565ae0e82b205783619" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43307 } } peers { permanent_uuid: "f5f38a0561514bd7880a57a0cd58e796" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 38895 } }
I20260806 03:41:51.929260 19276 heartbeater.cc:503] Master 127.15.252.126:43741 was elected leader, sending a full tablet report...
I20260806 03:41:51.929546 19452 ts_tablet_manager.cc:1428] T dec6183f982c4fe9adfae4666cc68eee P f5f38a0561514bd7880a57a0cd58e796: Time spent starting tablet: real 0.008s user 0.004s sys 0.005s
W20260806 03:41:51.929891 19452 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
W20260806 03:41:52.005923 19434 tablet.cc:2369] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:41:52.218164 19456 raft_consensus.cc:482] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:41:52.218864 19456 raft_consensus.cc:504] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3398149bf24815ae2e76918bd0b27c" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 38021 } } peers { permanent_uuid: "c2f74e482ee44565ae0e82b205783619" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43307 } } peers { permanent_uuid: "f5f38a0561514bd7880a57a0cd58e796" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 38895 } }
I20260806 03:41:52.220875 19456 leader_election.cc:288] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c2f74e482ee44565ae0e82b205783619 (127.15.252.67:43307), f5f38a0561514bd7880a57a0cd58e796 (127.15.252.65:38895)
I20260806 03:41:52.247625 19251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dec6183f982c4fe9adfae4666cc68eee" candidate_uuid: "4e3398149bf24815ae2e76918bd0b27c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f38a0561514bd7880a57a0cd58e796" is_pre_election: true
I20260806 03:41:52.248303 19405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dec6183f982c4fe9adfae4666cc68eee" candidate_uuid: "4e3398149bf24815ae2e76918bd0b27c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2f74e482ee44565ae0e82b205783619" is_pre_election: true
I20260806 03:41:52.248512 19251 raft_consensus.cc:2454] T dec6183f982c4fe9adfae4666cc68eee P f5f38a0561514bd7880a57a0cd58e796 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e3398149bf24815ae2e76918bd0b27c in term 0.
I20260806 03:41:52.249040 19405 raft_consensus.cc:2454] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e3398149bf24815ae2e76918bd0b27c in term 0.
I20260806 03:41:52.249886 19290 leader_election.cc:302] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e3398149bf24815ae2e76918bd0b27c, f5f38a0561514bd7880a57a0cd58e796; no voters:
I20260806 03:41:52.250918 19456 raft_consensus.cc:2781] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:41:52.251227 19456 raft_consensus.cc:482] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:41:52.251482 19456 raft_consensus.cc:3037] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:52.257987 19456 raft_consensus.cc:504] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3398149bf24815ae2e76918bd0b27c" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 38021 } } peers { permanent_uuid: "c2f74e482ee44565ae0e82b205783619" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43307 } } peers { permanent_uuid: "f5f38a0561514bd7880a57a0cd58e796" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 38895 } }
I20260806 03:41:52.260449 19456 leader_election.cc:288] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c [CANDIDATE]: Term 1 election: Requested vote from peers c2f74e482ee44565ae0e82b205783619 (127.15.252.67:43307), f5f38a0561514bd7880a57a0cd58e796 (127.15.252.65:38895)
I20260806 03:41:52.261390 19405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dec6183f982c4fe9adfae4666cc68eee" candidate_uuid: "4e3398149bf24815ae2e76918bd0b27c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c2f74e482ee44565ae0e82b205783619"
I20260806 03:41:52.261685 19251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dec6183f982c4fe9adfae4666cc68eee" candidate_uuid: "4e3398149bf24815ae2e76918bd0b27c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5f38a0561514bd7880a57a0cd58e796"
I20260806 03:41:52.262101 19405 raft_consensus.cc:3037] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:52.262461 19251 raft_consensus.cc:3037] T dec6183f982c4fe9adfae4666cc68eee P f5f38a0561514bd7880a57a0cd58e796 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:52.268602 19251 raft_consensus.cc:2454] T dec6183f982c4fe9adfae4666cc68eee P f5f38a0561514bd7880a57a0cd58e796 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e3398149bf24815ae2e76918bd0b27c in term 1.
I20260806 03:41:52.269454 19405 raft_consensus.cc:2454] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e3398149bf24815ae2e76918bd0b27c in term 1.
I20260806 03:41:52.269851 19290 leader_election.cc:302] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e3398149bf24815ae2e76918bd0b27c, f5f38a0561514bd7880a57a0cd58e796; no voters:
I20260806 03:41:52.271416 19456 raft_consensus.cc:2781] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:52.272562 19456 raft_consensus.cc:686] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c [term 1 LEADER]: Becoming Leader. State: Replica: 4e3398149bf24815ae2e76918bd0b27c, State: Running, Role: LEADER
I20260806 03:41:52.273398 19456 consensus_queue.cc:237] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3398149bf24815ae2e76918bd0b27c" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 38021 } } peers { permanent_uuid: "c2f74e482ee44565ae0e82b205783619" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43307 } } peers { permanent_uuid: "f5f38a0561514bd7880a57a0cd58e796" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 38895 } }
I20260806 03:41:52.282467 19145 catalog_manager.cc:5495] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e3398149bf24815ae2e76918bd0b27c (127.15.252.66). New cstate: current_term: 1 leader_uuid: "4e3398149bf24815ae2e76918bd0b27c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3398149bf24815ae2e76918bd0b27c" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 38021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c2f74e482ee44565ae0e82b205783619" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43307 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5f38a0561514bd7880a57a0cd58e796" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 38895 } health_report { overall_health: UNKNOWN } } }
I20260806 03:41:52.391491 19395 tablet_service.cc:1514] Processing DeleteTablet for tablet dec6183f982c4fe9adfae4666cc68eee with delete_type TABLET_DATA_TOMBSTONED from {username='slave'} at 127.0.0.1:49204
I20260806 03:41:52.393438 19464 tablet_replica.cc:330] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: stopping tablet replica
I20260806 03:41:52.394192 19464 raft_consensus.cc:2229] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:41:52.394769 19464 raft_consensus.cc:2258] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:52.397188 19464 ts_tablet_manager.cc:1905] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260806 03:41:52.409143 19464 ts_tablet_manager.cc:1918] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 0.0
I20260806 03:41:52.409494 19464 log.cc:1196] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: Deleting WAL directory at /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/ts-2-root/wals/dec6183f982c4fe9adfae4666cc68eee
I20260806 03:41:52.414450 19465 ts_tablet_manager.cc:927] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: Initiating tablet copy from peer 4e3398149bf24815ae2e76918bd0b27c (127.15.252.66:38021)
I20260806 03:41:52.416376 19465 tablet_copy_client.cc:316] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: tablet copy: Beginning tablet copy session from remote peer at address 127.15.252.66:38021
I20260806 03:41:52.430195 19340 tablet_copy_service.cc:140] P 4e3398149bf24815ae2e76918bd0b27c: Received BeginTabletCopySession request for tablet dec6183f982c4fe9adfae4666cc68eee from peer c2f74e482ee44565ae0e82b205783619 ({username='slave'} at 127.0.0.1:53274)
I20260806 03:41:52.430886 19340 tablet_copy_service.cc:161] P 4e3398149bf24815ae2e76918bd0b27c: Beginning new tablet copy session on tablet dec6183f982c4fe9adfae4666cc68eee from peer c2f74e482ee44565ae0e82b205783619 at {username='slave'} at 127.0.0.1:53274: session id = c2f74e482ee44565ae0e82b205783619-dec6183f982c4fe9adfae4666cc68eee
I20260806 03:41:52.437223 19340 tablet_copy_source_session.cc:205] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 03:41:52.440174 19465 ts_tablet_manager.cc:1905] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260806 03:41:52.451663 19465 ts_tablet_manager.cc:1918] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 0.0
I20260806 03:41:52.452332 19465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dec6183f982c4fe9adfae4666cc68eee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:52.457949 19465 tablet_copy_client.cc:791] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: tablet copy: Starting download of 0 data blocks...
I20260806 03:41:52.458564 19465 tablet_copy_client.cc:655] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: tablet copy: Starting download of 1 WAL segments...
I20260806 03:41:52.460631 19145 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51400:
table { table_id: "3955dc6c9edb4d5893cc40be0d696781" table_name: "test-table" }
I20260806 03:41:52.461385 19145 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51400:
table { table_id: "3955dc6c9edb4d5893cc40be0d696781" table_name: "test-table" }
I20260806 03:41:52.463341 19465 tablet_copy_client.cc:531] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 03:41:52.471508 19465 tablet_bootstrap.cc:492] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: Bootstrap starting.
I20260806 03:41:52.472398 19145 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2: Sending DeleteTablet for 3 replicas of tablet dec6183f982c4fe9adfae4666cc68eee
I20260806 03:41:52.475075 19318 tablet_service.cc:1514] Processing DeleteTablet for tablet dec6183f982c4fe9adfae4666cc68eee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:41:52 UTC) from {username='slave'} at 127.0.0.1:53252
I20260806 03:41:52.475308 19395 tablet_service.cc:1514] Processing DeleteTablet for tablet dec6183f982c4fe9adfae4666cc68eee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:41:52 UTC) from {username='slave'} at 127.0.0.1:49184
I20260806 03:41:52.475361 19241 tablet_service.cc:1514] Processing DeleteTablet for tablet dec6183f982c4fe9adfae4666cc68eee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:41:52 UTC) from {username='slave'} at 127.0.0.1:59550
W20260806 03:41:52.477370 19128 catalog_manager.cc:4821] TS c2f74e482ee44565ae0e82b205783619 (127.15.252.67:43307): delete failed for tablet dec6183f982c4fe9adfae4666cc68eee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dec6183f982c4fe9adfae4666cc68eee already in progress: copying tablet
I20260806 03:41:52.479743 19468 tablet_replica.cc:330] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c: stopping tablet replica
I20260806 03:41:52.479892 19469 tablet_replica.cc:330] T dec6183f982c4fe9adfae4666cc68eee P f5f38a0561514bd7880a57a0cd58e796: stopping tablet replica
I20260806 03:41:52.480501 19468 raft_consensus.cc:2229] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:41:52.481392 19468 pending_rounds.cc:62] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c: Trying to abort 1 pending ops.
I20260806 03:41:52.481654 19469 raft_consensus.cc:2229] T dec6183f982c4fe9adfae4666cc68eee P f5f38a0561514bd7880a57a0cd58e796 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:41:52.481702 19468 pending_rounds.cc:68] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315405669480660992 op_type: NO_OP noop_request { }
I20260806 03:41:52.482395 19469 raft_consensus.cc:2258] T dec6183f982c4fe9adfae4666cc68eee P f5f38a0561514bd7880a57a0cd58e796 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:52.482379 19468 raft_consensus.cc:2866] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260806 03:41:52.483175 19468 raft_consensus.cc:2258] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:52.485028 19469 ts_tablet_manager.cc:1905] T dec6183f982c4fe9adfae4666cc68eee P f5f38a0561514bd7880a57a0cd58e796: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:41:52.485987 19468 ts_tablet_manager.cc:1905] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:41:52.491190 19465 tablet_bootstrap.cc:492] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260806 03:41:52.492250 19465 tablet_bootstrap.cc:492] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: Bootstrap complete.
I20260806 03:41:52.493000 19465 ts_tablet_manager.cc:1397] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: Time spent bootstrapping tablet: real 0.022s user 0.017s sys 0.001s
I20260806 03:41:52.495958 19465 raft_consensus.cc:348] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3398149bf24815ae2e76918bd0b27c" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 38021 } } peers { permanent_uuid: "c2f74e482ee44565ae0e82b205783619" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43307 } } peers { permanent_uuid: "f5f38a0561514bd7880a57a0cd58e796" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 38895 } }
I20260806 03:41:52.496950 19465 raft_consensus.cc:729] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2f74e482ee44565ae0e82b205783619, State: Initialized, Role: FOLLOWER
I20260806 03:41:52.497289 19469 ts_tablet_manager.cc:1918] T dec6183f982c4fe9adfae4666cc68eee P f5f38a0561514bd7880a57a0cd58e796: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0
I20260806 03:41:52.497694 19469 log.cc:1196] T dec6183f982c4fe9adfae4666cc68eee P f5f38a0561514bd7880a57a0cd58e796: Deleting WAL directory at /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/ts-0-root/wals/dec6183f982c4fe9adfae4666cc68eee
I20260806 03:41:52.498379 19469 ts_tablet_manager.cc:1939] T dec6183f982c4fe9adfae4666cc68eee P f5f38a0561514bd7880a57a0cd58e796: Deleting consensus metadata
I20260806 03:41:52.497761 19465 consensus_queue.cc:260] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e3398149bf24815ae2e76918bd0b27c" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 38021 } } peers { permanent_uuid: "c2f74e482ee44565ae0e82b205783619" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43307 } } peers { permanent_uuid: "f5f38a0561514bd7880a57a0cd58e796" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 38895 } }
I20260806 03:41:52.500797 19465 ts_tablet_manager.cc:1428] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: Time spent starting tablet: real 0.007s user 0.005s sys 0.004s
W20260806 03:41:52.501091 19465 ts_tablet_manager.cc:1450] Injecting 50ms of latency into OpenTablet
I20260806 03:41:52.501742 19129 catalog_manager.cc:4841] TS f5f38a0561514bd7880a57a0cd58e796 (127.15.252.65:38895): tablet dec6183f982c4fe9adfae4666cc68eee (table test-table [id=3955dc6c9edb4d5893cc40be0d696781]) successfully deleted
I20260806 03:41:52.502413 19468 ts_tablet_manager.cc:1918] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 03:41:52.502800 19468 log.cc:1196] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c: Deleting WAL directory at /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/ts-1-root/wals/dec6183f982c4fe9adfae4666cc68eee
I20260806 03:41:52.503404 19468 ts_tablet_manager.cc:1939] T dec6183f982c4fe9adfae4666cc68eee P 4e3398149bf24815ae2e76918bd0b27c: Deleting consensus metadata
I20260806 03:41:52.504647 19130 catalog_manager.cc:4841] TS 4e3398149bf24815ae2e76918bd0b27c (127.15.252.66:38021): tablet dec6183f982c4fe9adfae4666cc68eee (table test-table [id=3955dc6c9edb4d5893cc40be0d696781]) successfully deleted
I20260806 03:41:52.506310 19395 tablet_service.cc:1514] Processing DeleteTablet for tablet dec6183f982c4fe9adfae4666cc68eee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:41:52 UTC) from {username='slave'} at 127.0.0.1:49184
W20260806 03:41:52.507707 19128 catalog_manager.cc:4821] TS c2f74e482ee44565ae0e82b205783619 (127.15.252.67:43307): delete failed for tablet dec6183f982c4fe9adfae4666cc68eee because tablet deleting was already in progress. No further retry: Already present: State transition of tablet dec6183f982c4fe9adfae4666cc68eee already in progress: copying tablet
I20260806 03:41:52.553814 19340 tablet_copy_service.cc:342] P 4e3398149bf24815ae2e76918bd0b27c: Request end of tablet copy session c2f74e482ee44565ae0e82b205783619-dec6183f982c4fe9adfae4666cc68eee received from {username='slave'} at 127.0.0.1:53274
I20260806 03:41:52.554282 19340 tablet_copy_service.cc:433] P 4e3398149bf24815ae2e76918bd0b27c: ending tablet copy session c2f74e482ee44565ae0e82b205783619-dec6183f982c4fe9adfae4666cc68eee on tablet dec6183f982c4fe9adfae4666cc68eee with peer c2f74e482ee44565ae0e82b205783619
I20260806 03:41:52.558696 19395 tablet_service.cc:1514] Processing DeleteTablet for tablet dec6183f982c4fe9adfae4666cc68eee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:41:52 UTC) from {username='slave'} at 127.0.0.1:49184
I20260806 03:41:52.559726 19464 tablet_replica.cc:330] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: stopping tablet replica
I20260806 03:41:52.560527 19464 raft_consensus.cc:2229] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:41:52.560995 19464 pending_rounds.cc:62] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: Trying to abort 1 pending ops.
I20260806 03:41:52.561223 19464 pending_rounds.cc:68] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7315405669480660992 op_type: NO_OP noop_request { }
I20260806 03:41:52.561558 19464 raft_consensus.cc:2866] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260806 03:41:52.561878 19464 raft_consensus.cc:2258] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:52.564687 19464 ts_tablet_manager.cc:1905] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:41:52.576032 19464 ts_tablet_manager.cc:1918] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1
I20260806 03:41:52.576433 19464 log.cc:1196] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: Deleting WAL directory at /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TsTabletManagerITest.TestDeleteTableDuringTabletCopy.1785987678583700-16369-0/minicluster-data/ts-2-root/wals/dec6183f982c4fe9adfae4666cc68eee
I20260806 03:41:52.577153 19464 ts_tablet_manager.cc:1939] T dec6183f982c4fe9adfae4666cc68eee P c2f74e482ee44565ae0e82b205783619: Deleting consensus metadata
I20260806 03:41:52.579890 19128 catalog_manager.cc:4841] TS c2f74e482ee44565ae0e82b205783619 (127.15.252.67:43307): tablet dec6183f982c4fe9adfae4666cc68eee (table test-table [id=3955dc6c9edb4d5893cc40be0d696781]) successfully deleted
I20260806 03:41:52.610287 16369 tablet_server.cc:178] TabletServer@127.15.252.65:0 shutting down...
I20260806 03:41:52.634923 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:52.654160 16369 tablet_server.cc:194] TabletServer@127.15.252.65:0 shutdown complete.
I20260806 03:41:52.664120 16369 tablet_server.cc:178] TabletServer@127.15.252.66:0 shutting down...
I20260806 03:41:52.686966 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:52.706238 16369 tablet_server.cc:194] TabletServer@127.15.252.66:0 shutdown complete.
I20260806 03:41:52.716189 16369 tablet_server.cc:178] TabletServer@127.15.252.67:0 shutting down...
I20260806 03:41:52.742213 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:52.762028 16369 tablet_server.cc:194] TabletServer@127.15.252.67:0 shutdown complete.
I20260806 03:41:52.774314 16369 master.cc:537] Master@127.15.252.126:43741 shutting down...
I20260806 03:41:52.793809 16369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:41:52.794427 16369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:52.794826 16369 tablet_replica.cc:330] T 00000000000000000000000000000000 P 613e8a2f330d4f4dacbf5bb7e1a506f2: stopping tablet replica
I20260806 03:41:52.814966 16369 master.cc:552] Master@127.15.252.126:43741 shutdown complete.
[ OK ] TsTabletManagerITest.TestDeleteTableDuringTabletCopy (1647 ms)
[----------] 4 tests from TsTabletManagerITest (22656 ms total)
[----------] 4 tests from TxnStatusTabletManagementTest
[ RUN ] TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet
I20260806 03:41:52.840256 16369 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.252.126:36507
I20260806 03:41:52.841548 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:52.847214 19476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:52.848344 19477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:52.850339 19479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:52.851380 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:52.852238 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:52.852442 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:52.852663 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987712852648 us; error 0 us; skew 500 ppm
I20260806 03:41:52.853296 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:52.856353 16369 webserver.cc:460] Webserver started at http://127.15.252.126:44895/ using document root <none> and password file <none>
I20260806 03:41:52.856861 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:52.857023 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:52.857372 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:52.858529 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/master-0-root/instance:
uuid: "76ce3566e43444d3901d598af5b032dc"
format_stamp: "Formatted at 2026-08-06 03:41:52 on dist-test-slave-csg5"
I20260806 03:41:52.863487 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260806 03:41:52.867367 19484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:52.868407 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260806 03:41:52.868707 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/master-0-root
uuid: "76ce3566e43444d3901d598af5b032dc"
format_stamp: "Formatted at 2026-08-06 03:41:52 on dist-test-slave-csg5"
I20260806 03:41:52.868984 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:52.902230 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:52.903462 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:52.951547 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.126:36507
I20260806 03:41:52.951658 19537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.126:36507 every 8 connection(s)
I20260806 03:41:52.955821 19538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:52.966682 19538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc: Bootstrap starting.
I20260806 03:41:52.971530 19538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:52.976217 19538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc: No bootstrap required, opened a new log
I20260806 03:41:52.978578 19538 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76ce3566e43444d3901d598af5b032dc" member_type: VOTER }
I20260806 03:41:52.979218 19538 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:52.979704 19538 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76ce3566e43444d3901d598af5b032dc, State: Initialized, Role: FOLLOWER
I20260806 03:41:52.980568 19538 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76ce3566e43444d3901d598af5b032dc" member_type: VOTER }
I20260806 03:41:52.981047 19538 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:41:52.981268 19538 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:41:52.981545 19538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:52.986522 19538 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76ce3566e43444d3901d598af5b032dc" member_type: VOTER }
I20260806 03:41:52.987155 19538 leader_election.cc:302] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76ce3566e43444d3901d598af5b032dc; no voters:
I20260806 03:41:52.988498 19538 leader_election.cc:288] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:41:52.988926 19542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:52.990480 19542 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc [term 1 LEADER]: Becoming Leader. State: Replica: 76ce3566e43444d3901d598af5b032dc, State: Running, Role: LEADER
I20260806 03:41:52.991454 19538 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:41:52.991228 19542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76ce3566e43444d3901d598af5b032dc" member_type: VOTER }
I20260806 03:41:52.995662 19543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76ce3566e43444d3901d598af5b032dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76ce3566e43444d3901d598af5b032dc" member_type: VOTER } }
I20260806 03:41:52.995643 19544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76ce3566e43444d3901d598af5b032dc. Latest consensus state: current_term: 1 leader_uuid: "76ce3566e43444d3901d598af5b032dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76ce3566e43444d3901d598af5b032dc" member_type: VOTER } }
I20260806 03:41:52.996549 19543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:52.996636 19544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:53.000938 19548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:41:53.008972 19548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:41:53.010218 16369 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260806 03:41:53.018980 19548 catalog_manager.cc:1347] Generated new cluster ID: 2b04627e9e9d430ea412a1b96dfb82eb
I20260806 03:41:53.019321 19548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:41:53.037068 19548 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:41:53.039076 19548 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:41:53.060537 19548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc: Generated new TSK 0
I20260806 03:41:53.061420 19548 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:41:53.077562 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:53.084173 19560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:53.085976 19561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:53.087714 19563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:53.088793 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:53.089848 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:53.090051 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:53.090272 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987713090235 us; error 0 us; skew 500 ppm
I20260806 03:41:53.090934 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:53.093930 16369 webserver.cc:460] Webserver started at http://127.15.252.65:33967/ using document root <none> and password file <none>
I20260806 03:41:53.094441 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:53.094599 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:53.094952 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:53.096095 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-0-root/instance:
uuid: "cd0a09b19bba419694deda09a047a06b"
format_stamp: "Formatted at 2026-08-06 03:41:53 on dist-test-slave-csg5"
I20260806 03:41:53.101099 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 03:41:53.105001 19568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:53.105934 16369 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 03:41:53.106287 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-0-root
uuid: "cd0a09b19bba419694deda09a047a06b"
format_stamp: "Formatted at 2026-08-06 03:41:53 on dist-test-slave-csg5"
I20260806 03:41:53.106627 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:53.136826 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:53.138013 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:53.143093 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:53.143443 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:53.143836 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:53.144093 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:53.199759 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.65:36733
I20260806 03:41:53.199882 19643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.65:36733 every 8 connection(s)
I20260806 03:41:53.205220 19644 heartbeater.cc:347] Connected to a master server at 127.15.252.126:36507
I20260806 03:41:53.205880 19644 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:53.206800 19644 heartbeater.cc:511] Master 127.15.252.126:36507 requested a full tablet report, sending...
I20260806 03:41:53.209240 16369 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.006317815s
I20260806 03:41:53.209125 19501 ts_manager.cc:194] Registered new tserver with Master: cd0a09b19bba419694deda09a047a06b (127.15.252.65:36733)
I20260806 03:41:53.212049 19501 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37314
I20260806 03:41:53.253389 19606 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260806 03:41:53.255043 19606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:53.269002 19654 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b: Bootstrap starting.
I20260806 03:41:53.274085 19654 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:53.278574 19654 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b: No bootstrap required, opened a new log
I20260806 03:41:53.279157 19654 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.002s
I20260806 03:41:53.281167 19654 raft_consensus.cc:348] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "cd0a09b19bba419694deda09a047a06b" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 36733 } }
I20260806 03:41:53.281651 19654 raft_consensus.cc:374] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:53.281949 19654 raft_consensus.cc:729] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0a09b19bba419694deda09a047a06b, State: Initialized, Role: FOLLOWER
I20260806 03:41:53.282527 19654 consensus_queue.cc:260] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "cd0a09b19bba419694deda09a047a06b" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 36733 } }
I20260806 03:41:53.283059 19654 raft_consensus.cc:388] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:41:53.283303 19654 raft_consensus.cc:482] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:41:53.283581 19654 raft_consensus.cc:3037] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:53.288775 19654 raft_consensus.cc:504] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "cd0a09b19bba419694deda09a047a06b" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 36733 } }
I20260806 03:41:53.289407 19654 leader_election.cc:302] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd0a09b19bba419694deda09a047a06b; no voters:
I20260806 03:41:53.290712 19654 leader_election.cc:288] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:41:53.291165 19656 raft_consensus.cc:2781] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:53.292735 19656 raft_consensus.cc:686] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 1 LEADER]: Becoming Leader. State: Replica: cd0a09b19bba419694deda09a047a06b, State: Running, Role: LEADER
I20260806 03:41:53.293006 19644 heartbeater.cc:503] Master 127.15.252.126:36507 was elected leader, sending a full tablet report...
I20260806 03:41:53.293241 19654 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260806 03:41:53.293515 19656 consensus_queue.cc:237] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "cd0a09b19bba419694deda09a047a06b" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 36733 } }
I20260806 03:41:53.301379 19657 tablet_replica.cc:439] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd0a09b19bba419694deda09a047a06b" committed_config { opid_index: -1 peers { permanent_uuid: "cd0a09b19bba419694deda09a047a06b" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 36733 } } }
I20260806 03:41:53.302201 19657 tablet_replica.cc:442] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:41:53.302033 19658 tablet_replica.cc:439] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b: TxnStatusTablet state changed. Reason: New leader cd0a09b19bba419694deda09a047a06b. Latest consensus state: current_term: 1 leader_uuid: "cd0a09b19bba419694deda09a047a06b" committed_config { opid_index: -1 peers { permanent_uuid: "cd0a09b19bba419694deda09a047a06b" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 36733 } } }
I20260806 03:41:53.302745 19658 tablet_replica.cc:442] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:41:53.304158 19661 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
W20260806 03:41:53.304495 19501 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260806 03:41:53.304812 19661 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260806 03:41:53.310041 19501 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260806 03:41:53.350459 16369 tablet_server.cc:178] TabletServer@127.15.252.65:0 shutting down...
I20260806 03:41:53.377796 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:53.379179 16369 tablet_replica.cc:330] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b: stopping tablet replica
I20260806 03:41:53.379896 16369 raft_consensus.cc:2229] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:41:53.380492 16369 raft_consensus.cc:2258] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:53.395846 16369 tablet_server.cc:194] TabletServer@127.15.252.65:0 shutdown complete.
I20260806 03:41:53.410215 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:53.418629 19664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:53.418838 19665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:53.420586 19668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:53.421510 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:53.423377 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:53.423660 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:53.423849 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987713423827 us; error 0 us; skew 500 ppm
I20260806 03:41:53.424525 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:53.427911 16369 webserver.cc:460] Webserver started at http://127.15.252.65:33967/ using document root <none> and password file <none>
I20260806 03:41:53.428633 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:53.428887 16369 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260806 03:41:53.433063 16369 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 03:41:53.436383 19673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:53.437453 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 03:41:53.437789 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-0-root
uuid: "cd0a09b19bba419694deda09a047a06b"
format_stamp: "Formatted at 2026-08-06 03:41:53 on dist-test-slave-csg5"
I20260806 03:41:53.438086 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:53.450937 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:53.452453 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:53.462241 19682 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260806 03:41:53.471081 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 03:41:53.471448 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.001s sys 0.001s
I20260806 03:41:53.471828 16369 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 03:41:53.478590 16369 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 03:41:53.478942 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.007s user 0.008s sys 0.000s
I20260806 03:41:53.479206 19682 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b: Bootstrap starting.
I20260806 03:41:53.526350 19682 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:41:53.527896 19682 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b: Bootstrap complete.
I20260806 03:41:53.528942 19682 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b: Time spent bootstrapping tablet: real 0.050s user 0.035s sys 0.011s
I20260806 03:41:53.532267 19682 raft_consensus.cc:348] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "cd0a09b19bba419694deda09a047a06b" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 36733 } }
I20260806 03:41:53.533174 19682 raft_consensus.cc:729] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd0a09b19bba419694deda09a047a06b, State: Initialized, Role: FOLLOWER
I20260806 03:41:53.534246 19682 consensus_queue.cc:260] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "cd0a09b19bba419694deda09a047a06b" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 36733 } }
I20260806 03:41:53.535279 19682 raft_consensus.cc:388] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:41:53.535748 19682 raft_consensus.cc:482] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:41:53.536298 19682 raft_consensus.cc:3037] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:41:53.547046 19682 raft_consensus.cc:504] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "cd0a09b19bba419694deda09a047a06b" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 36733 } }
I20260806 03:41:53.548884 19682 leader_election.cc:302] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd0a09b19bba419694deda09a047a06b; no voters:
I20260806 03:41:53.558591 19699 raft_consensus.cc:2781] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 2 FOLLOWER]: Leader election won for term 2
I20260806 03:41:53.559355 19682 leader_election.cc:288] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 03:41:53.563413 19682 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b: Time spent starting tablet: real 0.034s user 0.021s sys 0.008s
I20260806 03:41:53.564379 19699 raft_consensus.cc:686] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 2 LEADER]: Becoming Leader. State: Replica: cd0a09b19bba419694deda09a047a06b, State: Running, Role: LEADER
I20260806 03:41:53.565934 19699 consensus_queue.cc:237] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "cd0a09b19bba419694deda09a047a06b" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 36733 } }
I20260806 03:41:53.569283 19703 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cd0a09b19bba419694deda09a047a06b" committed_config { opid_index: -1 peers { permanent_uuid: "cd0a09b19bba419694deda09a047a06b" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 36733 } } }
I20260806 03:41:53.569931 19703 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260806 03:41:53.569491 19704 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: New leader cd0a09b19bba419694deda09a047a06b. Latest consensus state: current_term: 2 leader_uuid: "cd0a09b19bba419694deda09a047a06b" committed_config { opid_index: -1 peers { permanent_uuid: "cd0a09b19bba419694deda09a047a06b" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 36733 } } }
I20260806 03:41:53.570385 19704 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260806 03:41:53.585178 19708 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 03:41:53.586035 19708 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 03:41:53.631114 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.65:36733
I20260806 03:41:53.631215 19754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.65:36733 every 8 connection(s)
I20260806 03:41:53.640252 19755 heartbeater.cc:347] Connected to a master server at 127.15.252.126:36507
I20260806 03:41:53.640678 16369 tablet_server.cc:178] TabletServer@127.15.252.65:36733 shutting down...
I20260806 03:41:53.640717 19755 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:53.641816 19755 heartbeater.cc:511] Master 127.15.252.126:36507 requested a full tablet report, sending...
I20260806 03:41:53.645444 19501 ts_manager.cc:194] Re-registered known tserver with Master: cd0a09b19bba419694deda09a047a06b (127.15.252.65:36733)
W20260806 03:41:53.646104 19501 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260806 03:41:53.647974 19501 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37336
I20260806 03:41:53.651093 19755 heartbeater.cc:503] Master 127.15.252.126:36507 was elected leader, sending a full tablet report...
W20260806 03:41:53.654008 19501 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260806 03:41:53.681102 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:53.682191 16369 tablet_replica.cc:330] stopping tablet replica
I20260806 03:41:53.683094 16369 raft_consensus.cc:2229] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 2 LEADER]: Raft consensus shutting down.
I20260806 03:41:53.683686 16369 raft_consensus.cc:2258] T 11111111111111111111111111111111 P cd0a09b19bba419694deda09a047a06b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:53.698446 16369 tablet_server.cc:194] TabletServer@127.15.252.65:36733 shutdown complete.
I20260806 03:41:53.713819 16369 master.cc:537] Master@127.15.252.126:36507 shutting down...
I20260806 03:41:53.739099 16369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:41:53.739703 16369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:53.740069 16369 tablet_replica.cc:330] T 00000000000000000000000000000000 P 76ce3566e43444d3901d598af5b032dc: stopping tablet replica
I20260806 03:41:53.759606 16369 master.cc:552] Master@127.15.252.126:36507 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestBootstrapTransactionStatusTablet (948 ms)
[ RUN ] TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet
I20260806 03:41:53.788993 16369 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.252.126:42617
I20260806 03:41:53.790176 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:53.796619 19761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:53.796778 19762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:53.806025 16369 server_base.cc:1034] running on GCE node
W20260806 03:41:53.805989 19764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:53.807312 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:53.807500 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:53.807690 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987713807678 us; error 0 us; skew 500 ppm
I20260806 03:41:53.808265 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:53.811054 16369 webserver.cc:460] Webserver started at http://127.15.252.126:45757/ using document root <none> and password file <none>
I20260806 03:41:53.811602 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:53.811798 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:53.812079 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:53.813257 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/master-0-root/instance:
uuid: "f520697a13e74cdaa684b4042956dbc5"
format_stamp: "Formatted at 2026-08-06 03:41:53 on dist-test-slave-csg5"
I20260806 03:41:53.818300 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 03:41:53.822355 19769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:53.823275 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 03:41:53.823627 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/master-0-root
uuid: "f520697a13e74cdaa684b4042956dbc5"
format_stamp: "Formatted at 2026-08-06 03:41:53 on dist-test-slave-csg5"
I20260806 03:41:53.823961 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:53.855897 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:53.857074 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:53.915205 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.126:42617
I20260806 03:41:53.915326 19822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.126:42617 every 8 connection(s)
I20260806 03:41:53.920078 19823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:53.932673 19823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5: Bootstrap starting.
I20260806 03:41:53.937668 19823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:53.943220 19823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5: No bootstrap required, opened a new log
I20260806 03:41:53.945428 19823 raft_consensus.cc:348] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f520697a13e74cdaa684b4042956dbc5" member_type: VOTER }
I20260806 03:41:53.945866 19823 raft_consensus.cc:374] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:53.946177 19823 raft_consensus.cc:729] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f520697a13e74cdaa684b4042956dbc5, State: Initialized, Role: FOLLOWER
I20260806 03:41:53.946913 19823 consensus_queue.cc:260] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f520697a13e74cdaa684b4042956dbc5" member_type: VOTER }
I20260806 03:41:53.947443 19823 raft_consensus.cc:388] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:41:53.947690 19823 raft_consensus.cc:482] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:41:53.948005 19823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:53.953567 19823 raft_consensus.cc:504] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f520697a13e74cdaa684b4042956dbc5" member_type: VOTER }
I20260806 03:41:53.954210 19823 leader_election.cc:302] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f520697a13e74cdaa684b4042956dbc5; no voters:
I20260806 03:41:53.955874 19823 leader_election.cc:288] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:41:53.956301 19826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:53.958449 19826 raft_consensus.cc:686] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5 [term 1 LEADER]: Becoming Leader. State: Replica: f520697a13e74cdaa684b4042956dbc5, State: Running, Role: LEADER
I20260806 03:41:53.959336 19826 consensus_queue.cc:237] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f520697a13e74cdaa684b4042956dbc5" member_type: VOTER }
I20260806 03:41:53.960142 19823 sys_catalog.cc:564] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:41:53.964250 19827 sys_catalog.cc:455] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f520697a13e74cdaa684b4042956dbc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f520697a13e74cdaa684b4042956dbc5" member_type: VOTER } }
I20260806 03:41:53.964865 19827 sys_catalog.cc:458] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5 [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:53.966249 19828 sys_catalog.cc:455] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f520697a13e74cdaa684b4042956dbc5. Latest consensus state: current_term: 1 leader_uuid: "f520697a13e74cdaa684b4042956dbc5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f520697a13e74cdaa684b4042956dbc5" member_type: VOTER } }
I20260806 03:41:53.966837 19828 sys_catalog.cc:458] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5 [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:53.978956 19832 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:41:53.988749 19832 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:41:53.992693 16369 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260806 03:41:54.000823 19832 catalog_manager.cc:1347] Generated new cluster ID: f281d9093427463db407841779882925
I20260806 03:41:54.001386 19832 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:41:54.017647 19832 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:41:54.020181 19832 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:41:54.042681 19832 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5: Generated new TSK 0
I20260806 03:41:54.043468 19832 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:41:54.059612 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:54.066737 19845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:54.068567 19846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:54.072341 19848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:54.072696 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:54.073859 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:54.074102 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:54.074298 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987714074283 us; error 0 us; skew 500 ppm
I20260806 03:41:54.074936 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:54.078253 16369 webserver.cc:460] Webserver started at http://127.15.252.65:34013/ using document root <none> and password file <none>
I20260806 03:41:54.078845 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:54.079061 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:54.079342 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:54.080744 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-0-root/instance:
uuid: "44ec395b49664e90baa7248b7e17c46a"
format_stamp: "Formatted at 2026-08-06 03:41:54 on dist-test-slave-csg5"
I20260806 03:41:54.086611 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 03:41:54.090626 19853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:54.091799 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 03:41:54.092271 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-0-root
uuid: "44ec395b49664e90baa7248b7e17c46a"
format_stamp: "Formatted at 2026-08-06 03:41:54 on dist-test-slave-csg5"
I20260806 03:41:54.092574 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:54.108704 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:54.109906 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:54.121188 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:54.121641 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:54.122047 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:54.122320 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:54.208145 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.65:35991
I20260806 03:41:54.208256 19923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.65:35991 every 8 connection(s)
I20260806 03:41:54.213214 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:41:54.213567 19924 heartbeater.cc:347] Connected to a master server at 127.15.252.126:42617
I20260806 03:41:54.213972 19924 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:54.214980 19924 heartbeater.cc:511] Master 127.15.252.126:42617 requested a full tablet report, sending...
I20260806 03:41:54.217555 19786 ts_manager.cc:194] Registered new tserver with Master: 44ec395b49664e90baa7248b7e17c46a (127.15.252.65:35991)
I20260806 03:41:54.221295 19786 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56956
W20260806 03:41:54.221372 19926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:54.229981 19927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:54.237988 19929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:54.238551 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:54.239846 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:54.240056 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:54.240301 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987714240264 us; error 0 us; skew 500 ppm
I20260806 03:41:54.240911 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:54.244208 16369 webserver.cc:460] Webserver started at http://127.15.252.66:42463/ using document root <none> and password file <none>
I20260806 03:41:54.244805 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:54.245019 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:54.245379 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:54.246578 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-1-root/instance:
uuid: "dc36a46f59a84109bcde54ac45d37afe"
format_stamp: "Formatted at 2026-08-06 03:41:54 on dist-test-slave-csg5"
I20260806 03:41:54.252255 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 03:41:54.256536 19935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:54.257591 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 03:41:54.258003 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-1-root
uuid: "dc36a46f59a84109bcde54ac45d37afe"
format_stamp: "Formatted at 2026-08-06 03:41:54 on dist-test-slave-csg5"
I20260806 03:41:54.258364 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:54.279515 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:54.280769 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:54.287955 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:54.288317 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:54.288681 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:54.288882 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:54.379552 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.66:42765
I20260806 03:41:54.379665 20007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.66:42765 every 8 connection(s)
I20260806 03:41:54.385571 20008 heartbeater.cc:347] Connected to a master server at 127.15.252.126:42617
I20260806 03:41:54.386159 20008 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:54.387107 20008 heartbeater.cc:511] Master 127.15.252.126:42617 requested a full tablet report, sending...
I20260806 03:41:54.389247 19786 ts_manager.cc:194] Registered new tserver with Master: dc36a46f59a84109bcde54ac45d37afe (127.15.252.66:42765)
I20260806 03:41:54.389899 16369 internal_mini_cluster.cc:371] 2 TS(s) registered with all masters after 0.006342022s
I20260806 03:41:54.391712 19786 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56978
I20260806 03:41:54.435978 19889 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260806 03:41:54.437971 19889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:54.456408 20018 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a: Bootstrap starting.
I20260806 03:41:54.461246 20018 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:54.465951 20018 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a: No bootstrap required, opened a new log
I20260806 03:41:54.466408 20018 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260806 03:41:54.468607 20018 raft_consensus.cc:348] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "44ec395b49664e90baa7248b7e17c46a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35991 } }
I20260806 03:41:54.469084 20018 raft_consensus.cc:374] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:54.469308 20018 raft_consensus.cc:729] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44ec395b49664e90baa7248b7e17c46a, State: Initialized, Role: FOLLOWER
I20260806 03:41:54.469967 20018 consensus_queue.cc:260] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "44ec395b49664e90baa7248b7e17c46a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35991 } }
I20260806 03:41:54.470505 20018 raft_consensus.cc:388] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:41:54.470800 20018 raft_consensus.cc:482] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:41:54.471081 20018 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:54.476914 20018 raft_consensus.cc:504] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "44ec395b49664e90baa7248b7e17c46a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35991 } }
I20260806 03:41:54.477612 20018 leader_election.cc:302] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44ec395b49664e90baa7248b7e17c46a; no voters:
I20260806 03:41:54.479182 20018 leader_election.cc:288] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:41:54.479581 20021 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:54.481107 19924 heartbeater.cc:503] Master 127.15.252.126:42617 was elected leader, sending a full tablet report...
I20260806 03:41:54.482084 20018 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a: Time spent starting tablet: real 0.015s user 0.017s sys 0.000s
I20260806 03:41:54.482506 20021 raft_consensus.cc:686] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 1 LEADER]: Becoming Leader. State: Replica: 44ec395b49664e90baa7248b7e17c46a, State: Running, Role: LEADER
I20260806 03:41:54.483366 20021 consensus_queue.cc:237] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "44ec395b49664e90baa7248b7e17c46a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35991 } }
I20260806 03:41:54.486970 20022 tablet_replica.cc:439] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44ec395b49664e90baa7248b7e17c46a" committed_config { opid_index: -1 peers { permanent_uuid: "44ec395b49664e90baa7248b7e17c46a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35991 } } }
I20260806 03:41:54.487573 20022 tablet_replica.cc:442] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:41:54.489490 20023 tablet_replica.cc:439] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a: TxnStatusTablet state changed. Reason: New leader 44ec395b49664e90baa7248b7e17c46a. Latest consensus state: current_term: 1 leader_uuid: "44ec395b49664e90baa7248b7e17c46a" committed_config { opid_index: -1 peers { permanent_uuid: "44ec395b49664e90baa7248b7e17c46a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35991 } } }
I20260806 03:41:54.490464 20023 tablet_replica.cc:442] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a: This TxnStatusTablet replica's current role is: LEADER
W20260806 03:41:54.490696 19786 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260806 03:41:54.502836 19786 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260806 03:41:54.503870 20025 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 03:41:54.504420 20025 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 03:41:54.574337 20030 ts_tablet_manager.cc:927] T 11111111111111111111111111111111 P dc36a46f59a84109bcde54ac45d37afe: Initiating tablet copy from peer 44ec395b49664e90baa7248b7e17c46a (127.15.252.65:35991)
I20260806 03:41:54.576179 20030 tablet_copy_client.cc:316] T 11111111111111111111111111111111 P dc36a46f59a84109bcde54ac45d37afe: tablet copy: Beginning tablet copy session from remote peer at address 127.15.252.65:35991
I20260806 03:41:54.589115 19909 tablet_copy_service.cc:140] P 44ec395b49664e90baa7248b7e17c46a: Received BeginTabletCopySession request for tablet 11111111111111111111111111111111 from peer dc36a46f59a84109bcde54ac45d37afe ({username='slave'} at 127.0.0.1:58914)
I20260806 03:41:54.589874 19909 tablet_copy_service.cc:161] P 44ec395b49664e90baa7248b7e17c46a: Beginning new tablet copy session on tablet 11111111111111111111111111111111 from peer dc36a46f59a84109bcde54ac45d37afe at {username='slave'} at 127.0.0.1:58914: session id = dc36a46f59a84109bcde54ac45d37afe-11111111111111111111111111111111
I20260806 03:41:54.596925 19909 tablet_copy_source_session.cc:205] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 03:41:54.601415 20030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:54.615334 20030 tablet_copy_client.cc:791] T 11111111111111111111111111111111 P dc36a46f59a84109bcde54ac45d37afe: tablet copy: Starting download of 0 data blocks...
I20260806 03:41:54.616089 20030 tablet_copy_client.cc:655] T 11111111111111111111111111111111 P dc36a46f59a84109bcde54ac45d37afe: tablet copy: Starting download of 1 WAL segments...
I20260806 03:41:54.620402 20030 tablet_copy_client.cc:531] T 11111111111111111111111111111111 P dc36a46f59a84109bcde54ac45d37afe: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 03:41:54.628504 20030 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P dc36a46f59a84109bcde54ac45d37afe: Bootstrap starting.
I20260806 03:41:54.651935 20030 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P dc36a46f59a84109bcde54ac45d37afe: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:41:54.652733 20030 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P dc36a46f59a84109bcde54ac45d37afe: Bootstrap complete.
I20260806 03:41:54.653328 20030 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P dc36a46f59a84109bcde54ac45d37afe: Time spent bootstrapping tablet: real 0.025s user 0.016s sys 0.008s
I20260806 03:41:54.655369 20030 raft_consensus.cc:348] T 11111111111111111111111111111111 P dc36a46f59a84109bcde54ac45d37afe [term 1 NON_PARTICIPANT]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "44ec395b49664e90baa7248b7e17c46a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35991 } }
I20260806 03:41:54.655840 20030 raft_consensus.cc:729] T 11111111111111111111111111111111 P dc36a46f59a84109bcde54ac45d37afe [term 1 NON_PARTICIPANT]: Becoming Follower/Learner. State: Replica: dc36a46f59a84109bcde54ac45d37afe, State: Initialized, Role: NON_PARTICIPANT
I20260806 03:41:54.656373 20030 consensus_queue.cc:260] T 11111111111111111111111111111111 P dc36a46f59a84109bcde54ac45d37afe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "44ec395b49664e90baa7248b7e17c46a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35991 } }
I20260806 03:41:54.658278 20034 tablet_replica.cc:439] T 11111111111111111111111111111111 P dc36a46f59a84109bcde54ac45d37afe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 peers { permanent_uuid: "44ec395b49664e90baa7248b7e17c46a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35991 } } }
I20260806 03:41:54.658547 20008 heartbeater.cc:503] Master 127.15.252.126:42617 was elected leader, sending a full tablet report...
I20260806 03:41:54.658747 20030 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P dc36a46f59a84109bcde54ac45d37afe: Time spent starting tablet: real 0.005s user 0.007s sys 0.001s
I20260806 03:41:54.659014 20034 tablet_replica.cc:442] T 11111111111111111111111111111111 P dc36a46f59a84109bcde54ac45d37afe: This TxnStatusTablet replica's current role is: NON_PARTICIPANT
I20260806 03:41:54.660838 19909 tablet_copy_service.cc:342] P 44ec395b49664e90baa7248b7e17c46a: Request end of tablet copy session dc36a46f59a84109bcde54ac45d37afe-11111111111111111111111111111111 received from {username='slave'} at 127.0.0.1:58914
I20260806 03:41:54.661268 19909 tablet_copy_service.cc:433] P 44ec395b49664e90baa7248b7e17c46a: ending tablet copy session dc36a46f59a84109bcde54ac45d37afe-11111111111111111111111111111111 on tablet 11111111111111111111111111111111 with peer dc36a46f59a84109bcde54ac45d37afe
W20260806 03:41:54.661680 19786 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260806 03:41:54.666142 19786 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260806 03:41:54.680907 16369 tablet_server.cc:178] TabletServer@127.15.252.65:0 shutting down...
I20260806 03:41:54.707023 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:54.708153 16369 tablet_replica.cc:330] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a: stopping tablet replica
I20260806 03:41:54.708786 16369 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:41:54.709324 16369 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:54.733294 16369 tablet_server.cc:194] TabletServer@127.15.252.65:0 shutdown complete.
I20260806 03:41:54.745585 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:54.752591 20035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:54.753057 20036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:54.755366 20038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:54.756392 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:54.757592 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:54.757874 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:54.758087 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987714758053 us; error 0 us; skew 500 ppm
I20260806 03:41:54.758977 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:54.762866 16369 webserver.cc:460] Webserver started at http://127.15.252.65:34013/ using document root <none> and password file <none>
I20260806 03:41:54.763434 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:54.763623 16369 fs_manager.cc:362] Using existing metadata directory in first data directory
I20260806 03:41:54.768345 16369 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.005s
I20260806 03:41:54.772333 20043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:54.773380 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260806 03:41:54.773758 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-0-root
uuid: "44ec395b49664e90baa7248b7e17c46a"
format_stamp: "Formatted at 2026-08-06 03:41:54 on dist-test-slave-csg5"
I20260806 03:41:54.774142 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet.1785987678583700-16369-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:54.807559 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:54.808898 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:54.817085 20052 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260806 03:41:54.825515 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 03:41:54.825909 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.001s sys 0.000s
I20260806 03:41:54.826277 16369 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 03:41:54.835247 16369 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 03:41:54.835632 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.009s user 0.008s sys 0.000s
I20260806 03:41:54.835754 20052 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a: Bootstrap starting.
I20260806 03:41:54.892364 20052 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:41:54.894039 20052 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a: Bootstrap complete.
I20260806 03:41:54.895023 20052 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a: Time spent bootstrapping tablet: real 0.060s user 0.040s sys 0.015s
I20260806 03:41:54.898947 20052 raft_consensus.cc:348] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "44ec395b49664e90baa7248b7e17c46a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35991 } }
I20260806 03:41:54.899863 20052 raft_consensus.cc:729] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44ec395b49664e90baa7248b7e17c46a, State: Initialized, Role: FOLLOWER
I20260806 03:41:54.901050 20052 consensus_queue.cc:260] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "44ec395b49664e90baa7248b7e17c46a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35991 } }
I20260806 03:41:54.902019 20052 raft_consensus.cc:388] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:41:54.902421 20052 raft_consensus.cc:482] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:41:54.902942 20052 raft_consensus.cc:3037] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:41:54.913861 20052 raft_consensus.cc:504] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "44ec395b49664e90baa7248b7e17c46a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35991 } }
I20260806 03:41:54.914924 20052 leader_election.cc:302] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44ec395b49664e90baa7248b7e17c46a; no voters:
I20260806 03:41:54.917641 20052 leader_election.cc:288] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 03:41:54.917934 20084 raft_consensus.cc:2781] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 2 FOLLOWER]: Leader election won for term 2
I20260806 03:41:54.925606 20052 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a: Time spent starting tablet: real 0.030s user 0.022s sys 0.006s
I20260806 03:41:54.926338 20084 raft_consensus.cc:686] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 2 LEADER]: Becoming Leader. State: Replica: 44ec395b49664e90baa7248b7e17c46a, State: Running, Role: LEADER
I20260806 03:41:54.927598 20084 consensus_queue.cc:237] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "44ec395b49664e90baa7248b7e17c46a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35991 } }
I20260806 03:41:54.936559 20088 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: New leader 44ec395b49664e90baa7248b7e17c46a. Latest consensus state: current_term: 2 leader_uuid: "44ec395b49664e90baa7248b7e17c46a" committed_config { opid_index: -1 peers { permanent_uuid: "44ec395b49664e90baa7248b7e17c46a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35991 } } }
I20260806 03:41:54.937170 20088 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260806 03:41:54.954268 20087 tablet_replica.cc:439] TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "44ec395b49664e90baa7248b7e17c46a" committed_config { opid_index: -1 peers { permanent_uuid: "44ec395b49664e90baa7248b7e17c46a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 35991 } } }
I20260806 03:41:54.954958 20087 tablet_replica.cc:442] This TxnStatusTablet replica's current role is: LEADER
I20260806 03:41:54.956077 20098 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 03:41:54.956758 20098 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 03:41:54.978046 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.65:35991
I20260806 03:41:54.978190 20124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.65:35991 every 8 connection(s)
I20260806 03:41:54.984715 16369 tablet_server.cc:178] TabletServer@127.15.252.65:35991 shutting down...
I20260806 03:41:54.984275 20125 heartbeater.cc:347] Connected to a master server at 127.15.252.126:42617
I20260806 03:41:54.985706 20125 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:54.989418 20125 heartbeater.cc:511] Master 127.15.252.126:42617 requested a full tablet report, sending...
I20260806 03:41:54.993196 19786 ts_manager.cc:194] Re-registered known tserver with Master: 44ec395b49664e90baa7248b7e17c46a (127.15.252.65:35991)
W20260806 03:41:54.994822 19786 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260806 03:41:54.996368 19786 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57034
I20260806 03:41:55.000779 20125 heartbeater.cc:503] Master 127.15.252.126:42617 was elected leader, sending a full tablet report...
W20260806 03:41:55.008131 19786 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260806 03:41:55.019583 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:55.020628 16369 tablet_replica.cc:330] stopping tablet replica
I20260806 03:41:55.021377 16369 raft_consensus.cc:2229] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 2 LEADER]: Raft consensus shutting down.
I20260806 03:41:55.021950 16369 raft_consensus.cc:2258] T 11111111111111111111111111111111 P 44ec395b49664e90baa7248b7e17c46a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:55.039927 16369 tablet_server.cc:194] TabletServer@127.15.252.65:35991 shutdown complete.
I20260806 03:41:55.054468 16369 tablet_server.cc:178] TabletServer@127.15.252.66:0 shutting down...
I20260806 03:41:55.085245 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:55.086496 16369 tablet_replica.cc:330] T 11111111111111111111111111111111 P dc36a46f59a84109bcde54ac45d37afe: stopping tablet replica
I20260806 03:41:55.087039 16369 raft_consensus.cc:2229] T 11111111111111111111111111111111 P dc36a46f59a84109bcde54ac45d37afe [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260806 03:41:55.087476 16369 raft_consensus.cc:2258] T 11111111111111111111111111111111 P dc36a46f59a84109bcde54ac45d37afe [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260806 03:41:55.108520 16369 tablet_server.cc:194] TabletServer@127.15.252.66:0 shutdown complete.
I20260806 03:41:55.121454 16369 master.cc:537] Master@127.15.252.126:42617 shutting down...
I20260806 03:41:55.140729 16369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:41:55.141400 16369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:55.141791 16369 tablet_replica.cc:330] T 00000000000000000000000000000000 P f520697a13e74cdaa684b4042956dbc5: stopping tablet replica
I20260806 03:41:55.162456 16369 master.cc:552] Master@127.15.252.126:42617 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestCopyTransactionStatusTablet (1403 ms)
[ RUN ] TxnStatusTabletManagementTest.TestTabletServerProxyCalls
I20260806 03:41:55.192595 16369 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.252.126:33961
I20260806 03:41:55.193704 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:55.200203 20131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:55.201606 20132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:55.204047 20134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:55.204669 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:55.205660 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:55.205905 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:55.206077 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987715206064 us; error 0 us; skew 500 ppm
I20260806 03:41:55.206830 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:55.210033 16369 webserver.cc:460] Webserver started at http://127.15.252.126:39461/ using document root <none> and password file <none>
I20260806 03:41:55.210785 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:55.211024 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:55.211369 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:55.212936 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1785987678583700-16369-0/minicluster-data/master-0-root/instance:
uuid: "0ff4dd71d4b64d36a294d17a6bb599f3"
format_stamp: "Formatted at 2026-08-06 03:41:55 on dist-test-slave-csg5"
I20260806 03:41:55.219503 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260806 03:41:55.224275 20139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:55.225306 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260806 03:41:55.225666 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1785987678583700-16369-0/minicluster-data/master-0-root
uuid: "0ff4dd71d4b64d36a294d17a6bb599f3"
format_stamp: "Formatted at 2026-08-06 03:41:55 on dist-test-slave-csg5"
I20260806 03:41:55.226045 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1785987678583700-16369-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1785987678583700-16369-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1785987678583700-16369-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:55.252420 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:55.253888 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:55.308084 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.126:33961
I20260806 03:41:55.308162 20194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.126:33961 every 8 connection(s)
I20260806 03:41:55.314111 20195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:55.325872 20195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3: Bootstrap starting.
I20260806 03:41:55.331692 20195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:55.336515 20195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3: No bootstrap required, opened a new log
I20260806 03:41:55.339087 20195 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff4dd71d4b64d36a294d17a6bb599f3" member_type: VOTER }
I20260806 03:41:55.339650 20195 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:55.339916 20195 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ff4dd71d4b64d36a294d17a6bb599f3, State: Initialized, Role: FOLLOWER
I20260806 03:41:55.340487 20195 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff4dd71d4b64d36a294d17a6bb599f3" member_type: VOTER }
I20260806 03:41:55.340957 20195 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:41:55.341218 20195 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:41:55.341485 20195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:55.346280 20195 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff4dd71d4b64d36a294d17a6bb599f3" member_type: VOTER }
I20260806 03:41:55.346897 20195 leader_election.cc:302] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ff4dd71d4b64d36a294d17a6bb599f3; no voters:
I20260806 03:41:55.348294 20195 leader_election.cc:288] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:41:55.348670 20198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:55.350479 20198 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3 [term 1 LEADER]: Becoming Leader. State: Replica: 0ff4dd71d4b64d36a294d17a6bb599f3, State: Running, Role: LEADER
I20260806 03:41:55.351276 20198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff4dd71d4b64d36a294d17a6bb599f3" member_type: VOTER }
I20260806 03:41:55.351946 20195 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:41:55.354315 20200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ff4dd71d4b64d36a294d17a6bb599f3. Latest consensus state: current_term: 1 leader_uuid: "0ff4dd71d4b64d36a294d17a6bb599f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff4dd71d4b64d36a294d17a6bb599f3" member_type: VOTER } }
I20260806 03:41:55.355044 20200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3 [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:55.356344 20199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ff4dd71d4b64d36a294d17a6bb599f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ff4dd71d4b64d36a294d17a6bb599f3" member_type: VOTER } }
I20260806 03:41:55.356904 20199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3 [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:55.366618 20203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:41:55.375468 20203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:41:55.377936 16369 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260806 03:41:55.387135 20203 catalog_manager.cc:1347] Generated new cluster ID: f99c599964e741deaf1656985faa1b64
I20260806 03:41:55.387503 20203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:41:55.411273 20203 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:41:55.413422 20203 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:41:55.444535 20203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3: Generated new TSK 0
I20260806 03:41:55.445372 20203 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:41:55.509421 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:55.518903 20216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:55.522171 20217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:55.526609 20219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:55.526777 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:55.528955 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:55.529366 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:55.529601 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987715529583 us; error 0 us; skew 500 ppm
I20260806 03:41:55.530372 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:55.533778 16369 webserver.cc:460] Webserver started at http://127.15.252.65:44745/ using document root <none> and password file <none>
I20260806 03:41:55.534423 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:55.534698 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:55.535027 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:55.536520 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1785987678583700-16369-0/minicluster-data/ts-0-root/instance:
uuid: "e23be40c95fc4316a6b43ae19a608a21"
format_stamp: "Formatted at 2026-08-06 03:41:55 on dist-test-slave-csg5"
I20260806 03:41:55.542963 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260806 03:41:55.547672 20224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:55.548874 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 03:41:55.549320 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1785987678583700-16369-0/minicluster-data/ts-0-root
uuid: "e23be40c95fc4316a6b43ae19a608a21"
format_stamp: "Formatted at 2026-08-06 03:41:55 on dist-test-slave-csg5"
I20260806 03:41:55.549880 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1785987678583700-16369-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1785987678583700-16369-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCalls.1785987678583700-16369-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:55.578354 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:55.580147 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:55.595580 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:55.596055 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.001s sys 0.000s
I20260806 03:41:55.596339 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:55.596522 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:55.680349 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.65:34903
I20260806 03:41:55.680590 20298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.65:34903 every 8 connection(s)
I20260806 03:41:55.686352 20299 heartbeater.cc:347] Connected to a master server at 127.15.252.126:33961
I20260806 03:41:55.686882 20299 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:55.687690 20299 heartbeater.cc:511] Master 127.15.252.126:33961 requested a full tablet report, sending...
I20260806 03:41:55.690402 20157 ts_manager.cc:194] Registered new tserver with Master: e23be40c95fc4316a6b43ae19a608a21 (127.15.252.65:34903)
I20260806 03:41:55.690546 16369 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.0062589s
I20260806 03:41:55.693533 20157 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41690
I20260806 03:41:55.737818 20262 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260806 03:41:55.739367 20262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:55.753940 20309 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21: Bootstrap starting.
I20260806 03:41:55.759464 20309 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:55.764254 20309 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21: No bootstrap required, opened a new log
I20260806 03:41:55.764753 20309 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.005s
I20260806 03:41:55.766865 20309 raft_consensus.cc:348] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "e23be40c95fc4316a6b43ae19a608a21" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 34903 } }
I20260806 03:41:55.767352 20309 raft_consensus.cc:374] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:55.767576 20309 raft_consensus.cc:729] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e23be40c95fc4316a6b43ae19a608a21, State: Initialized, Role: FOLLOWER
I20260806 03:41:55.768164 20309 consensus_queue.cc:260] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "e23be40c95fc4316a6b43ae19a608a21" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 34903 } }
I20260806 03:41:55.768708 20309 raft_consensus.cc:388] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:41:55.768965 20309 raft_consensus.cc:482] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:41:55.769230 20309 raft_consensus.cc:3037] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:55.778185 20309 raft_consensus.cc:504] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "e23be40c95fc4316a6b43ae19a608a21" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 34903 } }
I20260806 03:41:55.779011 20309 leader_election.cc:302] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e23be40c95fc4316a6b43ae19a608a21; no voters:
I20260806 03:41:55.780679 20309 leader_election.cc:288] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:41:55.781217 20311 raft_consensus.cc:2781] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:55.782899 20299 heartbeater.cc:503] Master 127.15.252.126:33961 was elected leader, sending a full tablet report...
I20260806 03:41:55.783746 20309 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21: Time spent starting tablet: real 0.019s user 0.010s sys 0.010s
I20260806 03:41:55.784261 20311 raft_consensus.cc:686] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21 [term 1 LEADER]: Becoming Leader. State: Replica: e23be40c95fc4316a6b43ae19a608a21, State: Running, Role: LEADER
I20260806 03:41:55.784996 20311 consensus_queue.cc:237] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "e23be40c95fc4316a6b43ae19a608a21" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 34903 } }
I20260806 03:41:55.789078 20312 tablet_replica.cc:439] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e23be40c95fc4316a6b43ae19a608a21" committed_config { opid_index: -1 peers { permanent_uuid: "e23be40c95fc4316a6b43ae19a608a21" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 34903 } } }
I20260806 03:41:55.789793 20312 tablet_replica.cc:442] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21: This TxnStatusTablet replica's current role is: LEADER
W20260806 03:41:55.792043 20157 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260806 03:41:55.798681 20315 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 03:41:55.799317 20315 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 03:41:55.799333 20313 tablet_replica.cc:439] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21: TxnStatusTablet state changed. Reason: New leader e23be40c95fc4316a6b43ae19a608a21. Latest consensus state: current_term: 1 leader_uuid: "e23be40c95fc4316a6b43ae19a608a21" committed_config { opid_index: -1 peers { permanent_uuid: "e23be40c95fc4316a6b43ae19a608a21" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 34903 } } }
I20260806 03:41:55.800237 20313 tablet_replica.cc:442] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21: This TxnStatusTablet replica's current role is: LEADER
W20260806 03:41:55.802567 20157 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
W20260806 03:41:55.867964 20242 meta_cache.cc:788] Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260806 03:41:55.868345 20242 txn_status_manager.cc:243] Participant participant of txn 1 returned error for BEGIN_COMMIT op, aborting: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260806 03:41:55.868695 20242 txn_status_manager.cc:205] Scheduling write for ABORT_IN_PROGRESS for txn 1
I20260806 03:41:55.878459 20242 txn_status_manager.cc:336] Participant participant was not found for ABORT_TXN, proceeding as if op succeeded: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260806 03:41:55.930078 20242 txn_status_manager.cc:243] Participant participant of txn 2 returned error for BEGIN_COMMIT op, aborting: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
I20260806 03:41:55.930394 20242 txn_status_manager.cc:205] Scheduling write for ABORT_IN_PROGRESS for txn 2
I20260806 03:41:55.939638 20242 txn_status_manager.cc:336] Participant participant was not found for ABORT_TXN, proceeding as if op succeeded: Not found: LookupRpcById { tablet: 'participant', attempt: 1 } failed
W20260806 03:41:55.956581 20261 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:44152
W20260806 03:41:55.959172 20261 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:44152
W20260806 03:41:55.961647 20261 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:44152
W20260806 03:41:55.964031 20261 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:44152
W20260806 03:41:55.966225 20261 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:44152
W20260806 03:41:55.968456 20261 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='hughie'} at 127.0.0.1:44152
W20260806 03:41:55.981652 20261 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:44156
W20260806 03:41:55.984136 20261 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:44156
W20260806 03:41:55.986198 20261 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:44156
W20260806 03:41:55.988695 20261 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:44156
W20260806 03:41:55.990767 20261 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:44156
W20260806 03:41:55.992687 20261 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerAdminService.CoordinateTransaction from {username='butcher'} at 127.0.0.1:44156
I20260806 03:41:55.996832 16369 tablet_server.cc:178] TabletServer@127.15.252.65:0 shutting down...
I20260806 03:41:56.025667 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:56.026463 16369 tablet_replica.cc:330] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21: stopping tablet replica
I20260806 03:41:56.027320 16369 raft_consensus.cc:2229] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:41:56.027985 16369 raft_consensus.cc:2258] T 11111111111111111111111111111111 P e23be40c95fc4316a6b43ae19a608a21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:56.054538 16369 tablet_server.cc:194] TabletServer@127.15.252.65:0 shutdown complete.
I20260806 03:41:56.065829 16369 master.cc:537] Master@127.15.252.126:33961 shutting down...
I20260806 03:41:56.086942 16369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:41:56.087491 16369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:56.087771 16369 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0ff4dd71d4b64d36a294d17a6bb599f3: stopping tablet replica
I20260806 03:41:56.107430 16369 master.cc:552] Master@127.15.252.126:33961 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestTabletServerProxyCalls (942 ms)
[ RUN ] TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors
I20260806 03:41:56.134625 16369 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.252.126:39387
I20260806 03:41:56.135829 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:56.141847 20324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:56.143043 20325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:56.145591 20327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:56.145848 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:56.146921 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:56.147142 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:56.147326 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987716147309 us; error 0 us; skew 500 ppm
I20260806 03:41:56.147917 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:56.150849 16369 webserver.cc:460] Webserver started at http://127.15.252.126:45549/ using document root <none> and password file <none>
I20260806 03:41:56.151387 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:56.151615 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:56.151916 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:56.153126 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1785987678583700-16369-0/minicluster-data/master-0-root/instance:
uuid: "e9e497f3ebaf4d7eb9f240890f057a18"
format_stamp: "Formatted at 2026-08-06 03:41:56 on dist-test-slave-csg5"
I20260806 03:41:56.158535 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260806 03:41:56.162480 20332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:56.163489 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 03:41:56.163755 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1785987678583700-16369-0/minicluster-data/master-0-root
uuid: "e9e497f3ebaf4d7eb9f240890f057a18"
format_stamp: "Formatted at 2026-08-06 03:41:56 on dist-test-slave-csg5"
I20260806 03:41:56.164034 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1785987678583700-16369-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1785987678583700-16369-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1785987678583700-16369-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:56.187373 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:56.188431 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:56.231496 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.126:39387
I20260806 03:41:56.231725 20386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.126:39387 every 8 connection(s)
I20260806 03:41:56.236073 20387 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:56.247016 20387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18: Bootstrap starting.
I20260806 03:41:56.251324 20387 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:56.256036 20387 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18: No bootstrap required, opened a new log
I20260806 03:41:56.258052 20387 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e497f3ebaf4d7eb9f240890f057a18" member_type: VOTER }
I20260806 03:41:56.258455 20387 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:56.258782 20387 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9e497f3ebaf4d7eb9f240890f057a18, State: Initialized, Role: FOLLOWER
I20260806 03:41:56.259397 20387 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e497f3ebaf4d7eb9f240890f057a18" member_type: VOTER }
I20260806 03:41:56.259903 20387 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:41:56.260143 20387 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:41:56.260426 20387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:56.265645 20387 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e497f3ebaf4d7eb9f240890f057a18" member_type: VOTER }
I20260806 03:41:56.266265 20387 leader_election.cc:302] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9e497f3ebaf4d7eb9f240890f057a18; no voters:
I20260806 03:41:56.267532 20387 leader_election.cc:288] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:41:56.267930 20391 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:56.269477 20391 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18 [term 1 LEADER]: Becoming Leader. State: Replica: e9e497f3ebaf4d7eb9f240890f057a18, State: Running, Role: LEADER
I20260806 03:41:56.270167 20391 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e497f3ebaf4d7eb9f240890f057a18" member_type: VOTER }
I20260806 03:41:56.270743 20387 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:41:56.274096 20392 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9e497f3ebaf4d7eb9f240890f057a18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e497f3ebaf4d7eb9f240890f057a18" member_type: VOTER } }
I20260806 03:41:56.274930 20392 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18 [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:56.274708 20393 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9e497f3ebaf4d7eb9f240890f057a18. Latest consensus state: current_term: 1 leader_uuid: "e9e497f3ebaf4d7eb9f240890f057a18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9e497f3ebaf4d7eb9f240890f057a18" member_type: VOTER } }
I20260806 03:41:56.275673 20393 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18 [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:56.279234 20398 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:41:56.284890 20398 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:41:56.286985 16369 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260806 03:41:56.294391 20398 catalog_manager.cc:1347] Generated new cluster ID: 5f360d3472ca4629966a0fcf3a6dd986
I20260806 03:41:56.294721 20398 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:41:56.326272 20398 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:41:56.328156 20398 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:41:56.342543 20398 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18: Generated new TSK 0
I20260806 03:41:56.343387 20398 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:41:56.358074 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:56.365828 20410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:56.367352 20411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:56.369796 20413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:56.370590 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:56.371507 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:56.371832 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:56.371970 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987716371960 us; error 0 us; skew 500 ppm
I20260806 03:41:56.372536 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:56.380419 16369 webserver.cc:460] Webserver started at http://127.15.252.65:40647/ using document root <none> and password file <none>
I20260806 03:41:56.381016 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:56.381258 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:56.381553 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:56.382854 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1785987678583700-16369-0/minicluster-data/ts-0-root/instance:
uuid: "ac1ed38c9f404a77ad9865f2aadf7976"
format_stamp: "Formatted at 2026-08-06 03:41:56 on dist-test-slave-csg5"
I20260806 03:41:56.388379 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 03:41:56.392129 20418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:56.393098 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 03:41:56.393358 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1785987678583700-16369-0/minicluster-data/ts-0-root
uuid: "ac1ed38c9f404a77ad9865f2aadf7976"
format_stamp: "Formatted at 2026-08-06 03:41:56 on dist-test-slave-csg5"
I20260806 03:41:56.393616 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1785987678583700-16369-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1785987678583700-16369-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors.1785987678583700-16369-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:56.421801 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:56.422916 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:56.428421 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:56.428745 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:56.429078 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:56.429302 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:56.491290 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.65:45063
I20260806 03:41:56.491513 20495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.65:45063 every 8 connection(s)
I20260806 03:41:56.496984 20496 heartbeater.cc:347] Connected to a master server at 127.15.252.126:39387
I20260806 03:41:56.497424 20496 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:56.498229 20496 heartbeater.cc:511] Master 127.15.252.126:39387 requested a full tablet report, sending...
I20260806 03:41:56.500483 20352 ts_manager.cc:194] Registered new tserver with Master: ac1ed38c9f404a77ad9865f2aadf7976 (127.15.252.65:45063)
I20260806 03:41:56.501464 16369 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.006485556s
I20260806 03:41:56.502772 20352 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37832
I20260806 03:41:56.542325 20457 tablet_service.cc:1467] Processing CreateTablet for tablet 11111111111111111111111111111111 (TXN_STATUS_TABLE table=11111111111111111111111111111111_table_name [id=11111111111111111111111111111111_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260806 03:41:56.543756 20457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11111111111111111111111111111111. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:56.557543 20506 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976: Bootstrap starting.
I20260806 03:41:56.561885 20506 tablet_bootstrap.cc:654] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:56.566569 20506 tablet_bootstrap.cc:492] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976: No bootstrap required, opened a new log
I20260806 03:41:56.567054 20506 ts_tablet_manager.cc:1397] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260806 03:41:56.569169 20506 raft_consensus.cc:348] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "ac1ed38c9f404a77ad9865f2aadf7976" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 45063 } }
I20260806 03:41:56.569648 20506 raft_consensus.cc:374] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:56.570078 20506 raft_consensus.cc:729] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac1ed38c9f404a77ad9865f2aadf7976, State: Initialized, Role: FOLLOWER
I20260806 03:41:56.570968 20506 consensus_queue.cc:260] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "ac1ed38c9f404a77ad9865f2aadf7976" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 45063 } }
I20260806 03:41:56.571537 20506 raft_consensus.cc:388] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:41:56.571769 20506 raft_consensus.cc:482] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:41:56.572072 20506 raft_consensus.cc:3037] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:56.577636 20506 raft_consensus.cc:504] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "ac1ed38c9f404a77ad9865f2aadf7976" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 45063 } }
I20260806 03:41:56.578269 20506 leader_election.cc:302] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac1ed38c9f404a77ad9865f2aadf7976; no voters:
I20260806 03:41:56.579566 20506 leader_election.cc:288] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:41:56.580025 20508 raft_consensus.cc:2781] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:56.581714 20508 raft_consensus.cc:686] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976 [term 1 LEADER]: Becoming Leader. State: Replica: ac1ed38c9f404a77ad9865f2aadf7976, State: Running, Role: LEADER
I20260806 03:41:56.581979 20496 heartbeater.cc:503] Master 127.15.252.126:39387 was elected leader, sending a full tablet report...
I20260806 03:41:56.582275 20506 ts_tablet_manager.cc:1428] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260806 03:41:56.582516 20508 consensus_queue.cc:237] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "ac1ed38c9f404a77ad9865f2aadf7976" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 45063 } }
I20260806 03:41:56.586876 20510 tablet_replica.cc:439] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976: TxnStatusTablet state changed. Reason: New leader ac1ed38c9f404a77ad9865f2aadf7976. Latest consensus state: current_term: 1 leader_uuid: "ac1ed38c9f404a77ad9865f2aadf7976" committed_config { opid_index: -1 peers { permanent_uuid: "ac1ed38c9f404a77ad9865f2aadf7976" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 45063 } } }
I20260806 03:41:56.587110 20509 tablet_replica.cc:439] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac1ed38c9f404a77ad9865f2aadf7976" committed_config { opid_index: -1 peers { permanent_uuid: "ac1ed38c9f404a77ad9865f2aadf7976" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 45063 } } }
I20260806 03:41:56.587476 20510 tablet_replica.cc:442] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:41:56.587661 20509 tablet_replica.cc:442] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976: This TxnStatusTablet replica's current role is: LEADER
W20260806 03:41:56.588481 20352 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260806 03:41:56.591017 20514 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 03:41:56.591459 20514 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260806 03:41:56.593113 20352 catalog_manager.cc:5328] Ignoring report from unknown tablet 11111111111111111111111111111111
I20260806 03:41:56.620440 20457 tablet_service.cc:1467] Processing CreateTablet for tablet 22222222222222222222222222222222 (table=22222222222222222222222222222222_table_name [id=22222222222222222222222222222222_table_id]), partition=RANGE (txn_id, entry_type, identifier) PARTITION (0, -128, "") <= VALUES < (100, -128, "")
I20260806 03:41:56.621683 20457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22222222222222222222222222222222. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:56.634025 20506 tablet_bootstrap.cc:492] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976: Bootstrap starting.
I20260806 03:41:56.638886 20506 tablet_bootstrap.cc:654] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:56.643484 20506 tablet_bootstrap.cc:492] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976: No bootstrap required, opened a new log
I20260806 03:41:56.643966 20506 ts_tablet_manager.cc:1397] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260806 03:41:56.646006 20506 raft_consensus.cc:348] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 peers { permanent_uuid: "ac1ed38c9f404a77ad9865f2aadf7976" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 45063 } }
I20260806 03:41:56.646553 20506 raft_consensus.cc:374] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:56.646868 20506 raft_consensus.cc:729] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac1ed38c9f404a77ad9865f2aadf7976, State: Initialized, Role: FOLLOWER
I20260806 03:41:56.647511 20506 consensus_queue.cc:260] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "ac1ed38c9f404a77ad9865f2aadf7976" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 45063 } }
I20260806 03:41:56.648051 20506 raft_consensus.cc:388] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:41:56.648365 20506 raft_consensus.cc:482] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:41:56.648669 20506 raft_consensus.cc:3037] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:56.653903 20506 raft_consensus.cc:504] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 peers { permanent_uuid: "ac1ed38c9f404a77ad9865f2aadf7976" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 45063 } }
I20260806 03:41:56.654541 20506 leader_election.cc:302] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac1ed38c9f404a77ad9865f2aadf7976; no voters:
I20260806 03:41:56.655148 20506 leader_election.cc:288] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:41:56.655349 20508 raft_consensus.cc:2781] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:56.655887 20508 raft_consensus.cc:686] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976 [term 1 LEADER]: Becoming Leader. State: Replica: ac1ed38c9f404a77ad9865f2aadf7976, State: Running, Role: LEADER
I20260806 03:41:56.657063 20506 ts_tablet_manager.cc:1428] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260806 03:41:56.656684 20508 consensus_queue.cc:237] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 peers { permanent_uuid: "ac1ed38c9f404a77ad9865f2aadf7976" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 45063 } }
W20260806 03:41:56.663511 20352 catalog_manager.cc:5328] Ignoring report from unknown tablet 22222222222222222222222222222222
W20260806 03:41:56.668444 20352 catalog_manager.cc:5328] Ignoring report from unknown tablet 22222222222222222222222222222222
I20260806 03:41:56.689055 16369 tablet_server.cc:178] TabletServer@127.15.252.65:0 shutting down...
I20260806 03:41:56.716282 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:41:56.717216 16369 tablet_replica.cc:330] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976: stopping tablet replica
I20260806 03:41:56.717909 16369 raft_consensus.cc:2229] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:41:56.718413 16369 raft_consensus.cc:2258] T 22222222222222222222222222222222 P ac1ed38c9f404a77ad9865f2aadf7976 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:56.721283 16369 tablet_replica.cc:330] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976: stopping tablet replica
I20260806 03:41:56.721840 16369 raft_consensus.cc:2229] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:41:56.722325 16369 raft_consensus.cc:2258] T 11111111111111111111111111111111 P ac1ed38c9f404a77ad9865f2aadf7976 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:56.746286 16369 tablet_server.cc:194] TabletServer@127.15.252.65:0 shutdown complete.
I20260806 03:41:56.759166 16369 master.cc:537] Master@127.15.252.126:39387 shutting down...
I20260806 03:41:56.779095 16369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:41:56.779937 16369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:41:56.780637 16369 tablet_replica.cc:330] T 00000000000000000000000000000000 P e9e497f3ebaf4d7eb9f240890f057a18: stopping tablet replica
I20260806 03:41:56.800580 16369 master.cc:552] Master@127.15.252.126:39387 shutdown complete.
[ OK ] TxnStatusTabletManagementTest.TestTabletServerProxyCallErrors (692 ms)
[----------] 4 tests from TxnStatusTabletManagementTest (3986 ms total)
[----------] 2 tests from FailedTabletsAreReplacedITest
[ RUN ] FailedTabletsAreReplacedITest.OneReplica/0
I20260806 03:41:56.826155 16369 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.252.126:38131
I20260806 03:41:56.827291 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:56.832839 20523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:56.833765 20524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:56.836009 20527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:56.836068 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:56.837078 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:56.837282 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:56.837445 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987716837428 us; error 0 us; skew 500 ppm
I20260806 03:41:56.838008 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:56.840816 16369 webserver.cc:460] Webserver started at http://127.15.252.126:46313/ using document root <none> and password file <none>
I20260806 03:41:56.841312 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:56.841511 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:56.841775 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:56.843111 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/master-0-root/instance:
uuid: "e5b6c3cc1d444896945227446d577102"
format_stamp: "Formatted at 2026-08-06 03:41:56 on dist-test-slave-csg5"
I20260806 03:41:56.847918 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 03:41:56.851806 20533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:56.852739 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 03:41:56.853070 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/master-0-root
uuid: "e5b6c3cc1d444896945227446d577102"
format_stamp: "Formatted at 2026-08-06 03:41:56 on dist-test-slave-csg5"
I20260806 03:41:56.853425 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:56.890585 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:56.891813 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:56.934983 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.126:38131
I20260806 03:41:56.935098 20587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.126:38131 every 8 connection(s)
I20260806 03:41:56.939145 20588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:56.951370 20588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102: Bootstrap starting.
I20260806 03:41:56.956203 20588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:56.960777 20588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102: No bootstrap required, opened a new log
I20260806 03:41:56.962937 20588 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b6c3cc1d444896945227446d577102" member_type: VOTER }
I20260806 03:41:56.963346 20588 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:56.963629 20588 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5b6c3cc1d444896945227446d577102, State: Initialized, Role: FOLLOWER
I20260806 03:41:56.964277 20588 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b6c3cc1d444896945227446d577102" member_type: VOTER }
I20260806 03:41:56.964738 20588 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:41:56.964998 20588 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:41:56.965265 20588 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:56.970202 20588 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b6c3cc1d444896945227446d577102" member_type: VOTER }
I20260806 03:41:56.970796 20588 leader_election.cc:302] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5b6c3cc1d444896945227446d577102; no voters:
I20260806 03:41:56.972121 20588 leader_election.cc:288] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:41:56.972697 20592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:56.974236 20592 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102 [term 1 LEADER]: Becoming Leader. State: Replica: e5b6c3cc1d444896945227446d577102, State: Running, Role: LEADER
I20260806 03:41:56.975004 20592 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b6c3cc1d444896945227446d577102" member_type: VOTER }
I20260806 03:41:56.975653 20588 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:41:56.978243 20593 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5b6c3cc1d444896945227446d577102" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b6c3cc1d444896945227446d577102" member_type: VOTER } }
I20260806 03:41:56.979162 20593 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102 [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:56.978250 20594 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5b6c3cc1d444896945227446d577102. Latest consensus state: current_term: 1 leader_uuid: "e5b6c3cc1d444896945227446d577102" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5b6c3cc1d444896945227446d577102" member_type: VOTER } }
I20260806 03:41:56.979800 20594 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102 [sys.catalog]: This master's current role is: LEADER
I20260806 03:41:56.983106 20598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:41:56.991145 20598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:41:56.992269 16369 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260806 03:41:57.000223 20598 catalog_manager.cc:1347] Generated new cluster ID: 403ffebc6ae74782b079cfdd269e785b
I20260806 03:41:57.000558 20598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:41:57.014384 20598 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:41:57.016203 20598 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:41:57.034905 20598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102: Generated new TSK 0
I20260806 03:41:57.035727 20598 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:41:57.059643 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:57.066011 20610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:57.067195 20611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:57.068935 16369 server_base.cc:1034] running on GCE node
W20260806 03:41:57.069505 20613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:57.070614 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:57.070915 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:57.071043 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987717071034 us; error 0 us; skew 500 ppm
I20260806 03:41:57.071627 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:57.074764 16369 webserver.cc:460] Webserver started at http://127.15.252.65:40385/ using document root <none> and password file <none>
I20260806 03:41:57.075805 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:57.076014 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:57.076288 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:57.077694 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/ts-0-root/instance:
uuid: "13b8745d7ae147f086c1b80b1578c29c"
format_stamp: "Formatted at 2026-08-06 03:41:57 on dist-test-slave-csg5"
I20260806 03:41:57.082962 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 03:41:57.086781 20618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:57.087679 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 03:41:57.087994 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/ts-0-root
uuid: "13b8745d7ae147f086c1b80b1578c29c"
format_stamp: "Formatted at 2026-08-06 03:41:57 on dist-test-slave-csg5"
I20260806 03:41:57.088312 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:57.102159 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:57.103281 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:57.104861 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:57.107364 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:57.107589 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:57.107846 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:57.108011 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:57.149739 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.65:44921
I20260806 03:41:57.149861 20683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.65:44921 every 8 connection(s)
I20260806 03:41:57.154663 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:57.163122 20688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:57.163471 20689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:57.166941 20691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:57.167912 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:57.169097 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:41:57.169516 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:57.169682 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987717169672 us; error 0 us; skew 500 ppm
I20260806 03:41:57.170342 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:57.172863 20684 heartbeater.cc:347] Connected to a master server at 127.15.252.126:38131
I20260806 03:41:57.173393 20684 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:57.173882 16369 webserver.cc:460] Webserver started at http://127.15.252.66:40163/ using document root <none> and password file <none>
I20260806 03:41:57.174247 20684 heartbeater.cc:511] Master 127.15.252.126:38131 requested a full tablet report, sending...
I20260806 03:41:57.174433 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:57.174773 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:57.175060 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:57.176563 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/ts-1-root/instance:
uuid: "616c869f814e47a4b9ed79fdd4cb090e"
format_stamp: "Formatted at 2026-08-06 03:41:57 on dist-test-slave-csg5"
I20260806 03:41:57.176919 20550 ts_manager.cc:194] Registered new tserver with Master: 13b8745d7ae147f086c1b80b1578c29c (127.15.252.65:44921)
I20260806 03:41:57.179777 20550 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58818
I20260806 03:41:57.183046 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260806 03:41:57.186931 20696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:57.187832 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 03:41:57.188186 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/ts-1-root
uuid: "616c869f814e47a4b9ed79fdd4cb090e"
format_stamp: "Formatted at 2026-08-06 03:41:57 on dist-test-slave-csg5"
I20260806 03:41:57.188622 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:57.210561 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:57.212212 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:57.213903 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:57.216958 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:57.217247 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:57.217520 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:57.217703 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:57.264060 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.66:34083
I20260806 03:41:57.264213 20763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.66:34083 every 8 connection(s)
I20260806 03:41:57.269277 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:41:57.277459 20767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:57.279313 20768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:41:57.281450 20770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:41:57.283735 16369 server_base.cc:1034] running on GCE node
I20260806 03:41:57.284750 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260806 03:41:57.284893 20764 heartbeater.cc:347] Connected to a master server at 127.15.252.126:38131
W20260806 03:41:57.285080 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:41:57.285486 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987717285461 us; error 0 us; skew 500 ppm
I20260806 03:41:57.285514 20764 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:57.286135 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:41:57.286412 20764 heartbeater.cc:511] Master 127.15.252.126:38131 requested a full tablet report, sending...
I20260806 03:41:57.288789 20550 ts_manager.cc:194] Registered new tserver with Master: 616c869f814e47a4b9ed79fdd4cb090e (127.15.252.66:34083)
I20260806 03:41:57.289264 16369 webserver.cc:460] Webserver started at http://127.15.252.67:33051/ using document root <none> and password file <none>
I20260806 03:41:57.290052 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:41:57.290308 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:41:57.290521 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:41:57.291014 20550 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58828
I20260806 03:41:57.291720 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/ts-2-root/instance:
uuid: "75740c963c7b48c886f86c2693e1ca3d"
format_stamp: "Formatted at 2026-08-06 03:41:57 on dist-test-slave-csg5"
I20260806 03:41:57.296603 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 03:41:57.300407 20775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:57.301319 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 03:41:57.301648 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/ts-2-root
uuid: "75740c963c7b48c886f86c2693e1ca3d"
format_stamp: "Formatted at 2026-08-06 03:41:57 on dist-test-slave-csg5"
I20260806 03:41:57.302000 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:41:57.320055 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:41:57.321229 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:41:57.322984 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:41:57.325321 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:41:57.325559 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:57.325807 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:41:57.326011 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:41:57.370793 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.67:43441
I20260806 03:41:57.370939 20840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.67:43441 every 8 connection(s)
I20260806 03:41:57.388585 20841 heartbeater.cc:347] Connected to a master server at 127.15.252.126:38131
I20260806 03:41:57.389021 20841 heartbeater.cc:464] Registering TS with master...
I20260806 03:41:57.389823 20841 heartbeater.cc:511] Master 127.15.252.126:38131 requested a full tablet report, sending...
I20260806 03:41:57.392112 20550 ts_manager.cc:194] Registered new tserver with Master: 75740c963c7b48c886f86c2693e1ca3d (127.15.252.67:43441)
I20260806 03:41:57.393122 16369 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018399912s
I20260806 03:41:57.393834 20550 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48962
I20260806 03:41:57.417196 16369 test_util.cc:268] Using random seed: -1418180397
I20260806 03:41:57.443661 20550 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48982:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 03:41:57.487502 20806 tablet_service.cc:1467] Processing CreateTablet for tablet 12375a74ecf8476aa87de61ef2e8de70 (DEFAULT_TABLE table=test-workload [id=bd64710989e54e719b5a7dccc9b4aafa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:41:57.488947 20806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12375a74ecf8476aa87de61ef2e8de70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:57.505133 20724 tablet_service.cc:1467] Processing CreateTablet for tablet 12375a74ecf8476aa87de61ef2e8de70 (DEFAULT_TABLE table=test-workload [id=bd64710989e54e719b5a7dccc9b4aafa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:41:57.506392 20724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12375a74ecf8476aa87de61ef2e8de70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:57.507126 20862 tablet_bootstrap.cc:492] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d: Bootstrap starting.
I20260806 03:41:57.513316 20862 tablet_bootstrap.cc:654] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:57.515326 20647 tablet_service.cc:1467] Processing CreateTablet for tablet 12375a74ecf8476aa87de61ef2e8de70 (DEFAULT_TABLE table=test-workload [id=bd64710989e54e719b5a7dccc9b4aafa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:41:57.516683 20647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12375a74ecf8476aa87de61ef2e8de70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:57.518731 20862 tablet_bootstrap.cc:492] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d: No bootstrap required, opened a new log
I20260806 03:41:57.519397 20862 ts_tablet_manager.cc:1397] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d: Time spent bootstrapping tablet: real 0.012s user 0.012s sys 0.000s
I20260806 03:41:57.522131 20862 raft_consensus.cc:348] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75740c963c7b48c886f86c2693e1ca3d" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43441 } } peers { permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 } } peers { permanent_uuid: "616c869f814e47a4b9ed79fdd4cb090e" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 34083 } }
I20260806 03:41:57.523059 20862 raft_consensus.cc:374] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:57.523432 20862 raft_consensus.cc:729] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75740c963c7b48c886f86c2693e1ca3d, State: Initialized, Role: FOLLOWER
I20260806 03:41:57.523856 20864 tablet_bootstrap.cc:492] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: Bootstrap starting.
I20260806 03:41:57.524317 20862 consensus_queue.cc:260] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75740c963c7b48c886f86c2693e1ca3d" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43441 } } peers { permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 } } peers { permanent_uuid: "616c869f814e47a4b9ed79fdd4cb090e" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 34083 } }
I20260806 03:41:57.526993 20841 heartbeater.cc:503] Master 127.15.252.126:38131 was elected leader, sending a full tablet report...
I20260806 03:41:57.528160 20862 ts_tablet_manager.cc:1428] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260806 03:41:57.529515 20864 tablet_bootstrap.cc:654] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:57.535635 20864 tablet_bootstrap.cc:492] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: No bootstrap required, opened a new log
I20260806 03:41:57.536115 20864 ts_tablet_manager.cc:1397] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20260806 03:41:57.538563 20864 raft_consensus.cc:348] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75740c963c7b48c886f86c2693e1ca3d" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43441 } } peers { permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 } } peers { permanent_uuid: "616c869f814e47a4b9ed79fdd4cb090e" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 34083 } }
I20260806 03:41:57.539408 20864 raft_consensus.cc:374] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:57.539985 20864 raft_consensus.cc:729] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 616c869f814e47a4b9ed79fdd4cb090e, State: Initialized, Role: FOLLOWER
I20260806 03:41:57.540602 20867 tablet_bootstrap.cc:492] T 12375a74ecf8476aa87de61ef2e8de70 P 13b8745d7ae147f086c1b80b1578c29c: Bootstrap starting.
I20260806 03:41:57.540647 20864 consensus_queue.cc:260] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75740c963c7b48c886f86c2693e1ca3d" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43441 } } peers { permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 } } peers { permanent_uuid: "616c869f814e47a4b9ed79fdd4cb090e" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 34083 } }
I20260806 03:41:57.543162 20764 heartbeater.cc:503] Master 127.15.252.126:38131 was elected leader, sending a full tablet report...
I20260806 03:41:57.543887 20864 ts_tablet_manager.cc:1428] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260806 03:41:57.547812 20867 tablet_bootstrap.cc:654] T 12375a74ecf8476aa87de61ef2e8de70 P 13b8745d7ae147f086c1b80b1578c29c: Neither blocks nor log segments found. Creating new log.
I20260806 03:41:57.555456 20867 tablet_bootstrap.cc:492] T 12375a74ecf8476aa87de61ef2e8de70 P 13b8745d7ae147f086c1b80b1578c29c: No bootstrap required, opened a new log
I20260806 03:41:57.555984 20867 ts_tablet_manager.cc:1397] T 12375a74ecf8476aa87de61ef2e8de70 P 13b8745d7ae147f086c1b80b1578c29c: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.004s
I20260806 03:41:57.557919 20867 raft_consensus.cc:348] T 12375a74ecf8476aa87de61ef2e8de70 P 13b8745d7ae147f086c1b80b1578c29c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75740c963c7b48c886f86c2693e1ca3d" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43441 } } peers { permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 } } peers { permanent_uuid: "616c869f814e47a4b9ed79fdd4cb090e" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 34083 } }
I20260806 03:41:57.558514 20867 raft_consensus.cc:374] T 12375a74ecf8476aa87de61ef2e8de70 P 13b8745d7ae147f086c1b80b1578c29c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:41:57.558808 20867 raft_consensus.cc:729] T 12375a74ecf8476aa87de61ef2e8de70 P 13b8745d7ae147f086c1b80b1578c29c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13b8745d7ae147f086c1b80b1578c29c, State: Initialized, Role: FOLLOWER
I20260806 03:41:57.559350 20867 consensus_queue.cc:260] T 12375a74ecf8476aa87de61ef2e8de70 P 13b8745d7ae147f086c1b80b1578c29c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75740c963c7b48c886f86c2693e1ca3d" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43441 } } peers { permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 } } peers { permanent_uuid: "616c869f814e47a4b9ed79fdd4cb090e" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 34083 } }
I20260806 03:41:57.561404 20684 heartbeater.cc:503] Master 127.15.252.126:38131 was elected leader, sending a full tablet report...
I20260806 03:41:57.561508 20867 ts_tablet_manager.cc:1428] T 12375a74ecf8476aa87de61ef2e8de70 P 13b8745d7ae147f086c1b80b1578c29c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 03:41:57.566094 20865 raft_consensus.cc:482] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:41:57.566640 20865 raft_consensus.cc:504] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75740c963c7b48c886f86c2693e1ca3d" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43441 } } peers { permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 } } peers { permanent_uuid: "616c869f814e47a4b9ed79fdd4cb090e" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 34083 } }
I20260806 03:41:57.569150 20865 leader_election.cc:288] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13b8745d7ae147f086c1b80b1578c29c (127.15.252.65:44921), 616c869f814e47a4b9ed79fdd4cb090e (127.15.252.66:34083)
I20260806 03:41:57.584110 20657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12375a74ecf8476aa87de61ef2e8de70" candidate_uuid: "75740c963c7b48c886f86c2693e1ca3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13b8745d7ae147f086c1b80b1578c29c" is_pre_election: true
I20260806 03:41:57.584892 20657 raft_consensus.cc:2454] T 12375a74ecf8476aa87de61ef2e8de70 P 13b8745d7ae147f086c1b80b1578c29c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75740c963c7b48c886f86c2693e1ca3d in term 0.
I20260806 03:41:57.586407 20780 leader_election.cc:302] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13b8745d7ae147f086c1b80b1578c29c, 75740c963c7b48c886f86c2693e1ca3d; no voters:
I20260806 03:41:57.588110 20865 raft_consensus.cc:2781] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:41:57.588485 20865 raft_consensus.cc:482] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:41:57.588393 20735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12375a74ecf8476aa87de61ef2e8de70" candidate_uuid: "75740c963c7b48c886f86c2693e1ca3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "616c869f814e47a4b9ed79fdd4cb090e" is_pre_election: true
I20260806 03:41:57.589042 20865 raft_consensus.cc:3037] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:57.589321 20735 raft_consensus.cc:2454] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75740c963c7b48c886f86c2693e1ca3d in term 0.
I20260806 03:41:57.595553 20865 raft_consensus.cc:504] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75740c963c7b48c886f86c2693e1ca3d" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43441 } } peers { permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 } } peers { permanent_uuid: "616c869f814e47a4b9ed79fdd4cb090e" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 34083 } }
I20260806 03:41:57.597241 20865 leader_election.cc:288] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [CANDIDATE]: Term 1 election: Requested vote from peers 13b8745d7ae147f086c1b80b1578c29c (127.15.252.65:44921), 616c869f814e47a4b9ed79fdd4cb090e (127.15.252.66:34083)
I20260806 03:41:57.598201 20657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12375a74ecf8476aa87de61ef2e8de70" candidate_uuid: "75740c963c7b48c886f86c2693e1ca3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13b8745d7ae147f086c1b80b1578c29c"
I20260806 03:41:57.598429 20735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12375a74ecf8476aa87de61ef2e8de70" candidate_uuid: "75740c963c7b48c886f86c2693e1ca3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "616c869f814e47a4b9ed79fdd4cb090e"
I20260806 03:41:57.598752 20657 raft_consensus.cc:3037] T 12375a74ecf8476aa87de61ef2e8de70 P 13b8745d7ae147f086c1b80b1578c29c [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:57.599085 20735 raft_consensus.cc:3037] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:41:57.604346 20657 raft_consensus.cc:2454] T 12375a74ecf8476aa87de61ef2e8de70 P 13b8745d7ae147f086c1b80b1578c29c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75740c963c7b48c886f86c2693e1ca3d in term 1.
I20260806 03:41:57.604585 20735 raft_consensus.cc:2454] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75740c963c7b48c886f86c2693e1ca3d in term 1.
I20260806 03:41:57.605496 20780 leader_election.cc:302] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13b8745d7ae147f086c1b80b1578c29c, 75740c963c7b48c886f86c2693e1ca3d; no voters:
I20260806 03:41:57.606282 20865 raft_consensus.cc:2781] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:41:57.607348 20865 raft_consensus.cc:686] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [term 1 LEADER]: Becoming Leader. State: Replica: 75740c963c7b48c886f86c2693e1ca3d, State: Running, Role: LEADER
I20260806 03:41:57.608103 20865 consensus_queue.cc:237] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75740c963c7b48c886f86c2693e1ca3d" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43441 } } peers { permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 } } peers { permanent_uuid: "616c869f814e47a4b9ed79fdd4cb090e" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 34083 } }
I20260806 03:41:57.615929 20549 catalog_manager.cc:5495] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 75740c963c7b48c886f86c2693e1ca3d (127.15.252.67). New cstate: current_term: 1 leader_uuid: "75740c963c7b48c886f86c2693e1ca3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75740c963c7b48c886f86c2693e1ca3d" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43441 } } peers { permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 } } peers { permanent_uuid: "616c869f814e47a4b9ed79fdd4cb090e" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 34083 } } }
I20260806 03:41:57.923683 20647 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260806 03:41:57.929554 20724 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260806 03:41:57.932973 20806 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260806 03:41:58.126760 20902 consensus_queue.cc:1035] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "616c869f814e47a4b9ed79fdd4cb090e" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 34083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:41:58.149727 20905 consensus_queue.cc:1035] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
e5b6c3cc1d444896945227446d577102 | 127.15.252.126:38131 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
13b8745d7ae147f086c1b80b1578c29c | 127.15.252.65:44921 | HEALTHY | <none> | 0 | 0
616c869f814e47a4b9ed79fdd4cb090e | 127.15.252.66:34083 | HEALTHY | <none> | 0 | 0
75740c963c7b48c886f86c2693e1ca3d | 127.15.252.67:43441 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 12375a74ecf8476aa87de61ef2e8de70 of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
75740c963c7b48c886f86c2693e1ca3d (127.15.252.67:43441): RUNNING [LEADER]
13b8745d7ae147f086c1b80b1578c29c (127.15.252.65:44921): RUNNING
616c869f814e47a4b9ed79fdd4cb090e (127.15.252.66:34083): RUNNING
All reported replicas are:
A = 75740c963c7b48c886f86c2693e1ca3d
B = 13b8745d7ae147f086c1b80b1578c29c
C = 616c869f814e47a4b9ed79fdd4cb090e
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 1 | -1 | Yes
B | A B C | 1 | -1 | Yes
C | A B C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260806 03:41:58.168447 16369 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260806 03:41:58.562804 20647 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260806 03:41:58.571774 20806 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260806 03:41:58.581009 20724 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
e5b6c3cc1d444896945227446d577102 | 127.15.252.126:38131 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
13b8745d7ae147f086c1b80b1578c29c | 127.15.252.65:44921 | HEALTHY | <none> | 0 | 0
616c869f814e47a4b9ed79fdd4cb090e | 127.15.252.66:34083 | HEALTHY | <none> | 0 | 0
75740c963c7b48c886f86c2693e1ca3d | 127.15.252.67:43441 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260806 03:41:58.778755 16369 log_verifier.cc:126] Checking tablet 12375a74ecf8476aa87de61ef2e8de70
I20260806 03:41:58.850931 16369 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 12375a74ecf8476aa87de61ef2e8de70
I20260806 03:41:58.851532 16369 tablet_replica.cc:330] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: stopping tablet replica
I20260806 03:41:58.852082 16369 raft_consensus.cc:2229] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:41:58.852555 16369 raft_consensus.cc:2258] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 03:41:59.070515 20779 consensus_peers.cc:487] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d -> Peer 616c869f814e47a4b9ed79fdd4cb090e (127.15.252.66:34083): Couldn't send request to peer 616c869f814e47a4b9ed79fdd4cb090e. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260806 03:41:59.072218 20905 raft_consensus.cc:998] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d: Attempting to remove follower 616c869f814e47a4b9ed79fdd4cb090e from the Raft config. Reason: The tablet replica hosted on peer 616c869f814e47a4b9ed79fdd4cb090e has failed
I20260806 03:41:59.076592 20905 consensus_queue.cc:237] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "75740c963c7b48c886f86c2693e1ca3d" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43441 } } peers { permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 } }
I20260806 03:41:59.085664 20657 raft_consensus.cc:1261] T 12375a74ecf8476aa87de61ef2e8de70 P 13b8745d7ae147f086c1b80b1578c29c [term 1 FOLLOWER]: Refusing update from remote peer 75740c963c7b48c886f86c2693e1ca3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:41:59.087843 20951 consensus_queue.cc:1035] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260806 03:41:59.096280 20905 raft_consensus.cc:2932] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 616c869f814e47a4b9ed79fdd4cb090e (127.15.252.66) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "75740c963c7b48c886f86c2693e1ca3d" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43441 } } peers { permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 } } }
I20260806 03:41:59.098049 20657 raft_consensus.cc:2932] T 12375a74ecf8476aa87de61ef2e8de70 P 13b8745d7ae147f086c1b80b1578c29c [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER 616c869f814e47a4b9ed79fdd4cb090e (127.15.252.66) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "75740c963c7b48c886f86c2693e1ca3d" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43441 } } peers { permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 } } }
I20260806 03:41:59.115455 20550 catalog_manager.cc:5495] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d reported cstate change: config changed from index -1 to 2, VOTER 616c869f814e47a4b9ed79fdd4cb090e (127.15.252.66) evicted. New cstate: current_term: 1 leader_uuid: "75740c963c7b48c886f86c2693e1ca3d" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "75740c963c7b48c886f86c2693e1ca3d" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43441 } } peers { permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 } } }
I20260806 03:41:59.142181 20724 tablet_service.cc:1514] Processing DeleteTablet for tablet 12375a74ecf8476aa87de61ef2e8de70 with delete_type TABLET_DATA_TOMBSTONED (TS 616c869f814e47a4b9ed79fdd4cb090e not found in new config with opid_index 2) from {username='slave'} at 127.0.0.1:52736
I20260806 03:41:59.148136 20964 ts_tablet_manager.cc:1905] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260806 03:41:59.148059 20816 consensus_queue.cc:237] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "75740c963c7b48c886f86c2693e1ca3d" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43441 } } peers { permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 } } peers { permanent_uuid: "616c869f814e47a4b9ed79fdd4cb090e" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 34083 } }
W20260806 03:41:59.156162 20779 consensus_peers.cc:487] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d -> Peer 616c869f814e47a4b9ed79fdd4cb090e (127.15.252.66:34083): Couldn't send request to peer 616c869f814e47a4b9ed79fdd4cb090e. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: FAILED. This is attempt 1: this message will repeat every 5th retry.
I20260806 03:41:59.161883 20657 raft_consensus.cc:1261] T 12375a74ecf8476aa87de61ef2e8de70 P 13b8745d7ae147f086c1b80b1578c29c [term 1 FOLLOWER]: Refusing update from remote peer 75740c963c7b48c886f86c2693e1ca3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260806 03:41:59.165719 20951 consensus_queue.cc:1035] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260806 03:41:59.172598 20964 ts_tablet_manager.cc:1918] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260806 03:41:59.173373 20964 log.cc:1196] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: Deleting WAL directory at /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_0.1785987678583700-16369-0/minicluster-data/ts-1-root/wals/12375a74ecf8476aa87de61ef2e8de70
I20260806 03:41:59.176450 20536 catalog_manager.cc:4841] TS 616c869f814e47a4b9ed79fdd4cb090e (127.15.252.66:34083): tablet 12375a74ecf8476aa87de61ef2e8de70 (table test-workload [id=bd64710989e54e719b5a7dccc9b4aafa]) successfully deleted
I20260806 03:41:59.178995 20905 raft_consensus.cc:2932] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 616c869f814e47a4b9ed79fdd4cb090e (127.15.252.66) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "75740c963c7b48c886f86c2693e1ca3d" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43441 } } peers { permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 } } peers { permanent_uuid: "616c869f814e47a4b9ed79fdd4cb090e" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 34083 } } }
I20260806 03:41:59.191816 20657 raft_consensus.cc:2932] T 12375a74ecf8476aa87de61ef2e8de70 P 13b8745d7ae147f086c1b80b1578c29c [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, VOTER 616c869f814e47a4b9ed79fdd4cb090e (127.15.252.66) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "75740c963c7b48c886f86c2693e1ca3d" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43441 } } peers { permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 } } peers { permanent_uuid: "616c869f814e47a4b9ed79fdd4cb090e" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 34083 } } }
I20260806 03:41:59.205865 20537 catalog_manager.cc:5008] ChangeConfig:ADD_PEER:VOTER RPC for tablet 12375a74ecf8476aa87de61ef2e8de70 with cas_config_opid_index 2: ChangeConfig:ADD_PEER:VOTER succeeded (attempt 1)
I20260806 03:41:59.211241 20548 catalog_manager.cc:5495] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d reported cstate change: config changed from index 2 to 3, VOTER 616c869f814e47a4b9ed79fdd4cb090e (127.15.252.66) added. New cstate: current_term: 1 leader_uuid: "75740c963c7b48c886f86c2693e1ca3d" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "75740c963c7b48c886f86c2693e1ca3d" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43441 } } peers { permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 } } peers { permanent_uuid: "616c869f814e47a4b9ed79fdd4cb090e" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 34083 } } }
I20260806 03:41:59.234609 20647 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260806 03:41:59.240226 20724 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260806 03:41:59.241140 20806 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
e5b6c3cc1d444896945227446d577102 | 127.15.252.126:38131 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
13b8745d7ae147f086c1b80b1578c29c | 127.15.252.65:44921 | HEALTHY | <none> | 0 | 0
616c869f814e47a4b9ed79fdd4cb090e | 127.15.252.66:34083 | HEALTHY | <none> | 0 | 0
75740c963c7b48c886f86c2693e1ca3d | 127.15.252.67:43441 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 12375a74ecf8476aa87de61ef2e8de70 of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
75740c963c7b48c886f86c2693e1ca3d (127.15.252.67:43441): RUNNING [LEADER]
13b8745d7ae147f086c1b80b1578c29c (127.15.252.65:44921): RUNNING
All reported replicas are:
A = 75740c963c7b48c886f86c2693e1ca3d
B = 13b8745d7ae147f086c1b80b1578c29c
C = 616c869f814e47a4b9ed79fdd4cb090e
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A* B | | | Yes
A | A* B C | 1 | 3 | Yes
B | A* B C | 1 | 3 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 0
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 2
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260806 03:41:59.457846 16369 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260806 03:41:59.657229 20980 ts_tablet_manager.cc:927] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: Initiating tablet copy from peer 75740c963c7b48c886f86c2693e1ca3d (127.15.252.67:43441)
I20260806 03:41:59.659561 20980 tablet_copy_client.cc:316] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: tablet copy: Beginning tablet copy session from remote peer at address 127.15.252.67:43441
I20260806 03:41:59.675422 20826 tablet_copy_service.cc:140] P 75740c963c7b48c886f86c2693e1ca3d: Received BeginTabletCopySession request for tablet 12375a74ecf8476aa87de61ef2e8de70 from peer 616c869f814e47a4b9ed79fdd4cb090e ({username='slave'} at 127.0.0.1:56106)
I20260806 03:41:59.676071 20826 tablet_copy_service.cc:161] P 75740c963c7b48c886f86c2693e1ca3d: Beginning new tablet copy session on tablet 12375a74ecf8476aa87de61ef2e8de70 from peer 616c869f814e47a4b9ed79fdd4cb090e at {username='slave'} at 127.0.0.1:56106: session id = 616c869f814e47a4b9ed79fdd4cb090e-12375a74ecf8476aa87de61ef2e8de70
I20260806 03:41:59.684312 20826 tablet_copy_source_session.cc:205] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 03:41:59.687012 20980 ts_tablet_manager.cc:1905] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: Deleting tablet data with delete state TABLET_DATA_COPYING
I20260806 03:41:59.701797 20980 ts_tablet_manager.cc:1918] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.1
I20260806 03:41:59.702589 20980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12375a74ecf8476aa87de61ef2e8de70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:41:59.709359 20980 tablet_copy_client.cc:791] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: tablet copy: Starting download of 0 data blocks...
I20260806 03:41:59.709998 20980 tablet_copy_client.cc:655] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: tablet copy: Starting download of 1 WAL segments...
I20260806 03:41:59.713896 20980 tablet_copy_client.cc:531] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 03:41:59.723635 20980 tablet_bootstrap.cc:492] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: Bootstrap starting.
I20260806 03:41:59.746938 20980 tablet_bootstrap.cc:492] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:41:59.747907 20980 tablet_bootstrap.cc:492] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: Bootstrap complete.
I20260806 03:41:59.748704 20980 ts_tablet_manager.cc:1397] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: Time spent bootstrapping tablet: real 0.025s user 0.019s sys 0.004s
I20260806 03:41:59.751618 20980 raft_consensus.cc:348] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "75740c963c7b48c886f86c2693e1ca3d" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43441 } } peers { permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 } } peers { permanent_uuid: "616c869f814e47a4b9ed79fdd4cb090e" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 34083 } }
I20260806 03:41:59.752370 20980 raft_consensus.cc:729] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 616c869f814e47a4b9ed79fdd4cb090e, State: Initialized, Role: FOLLOWER
I20260806 03:41:59.753100 20980 consensus_queue.cc:260] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "75740c963c7b48c886f86c2693e1ca3d" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 43441 } } peers { permanent_uuid: "13b8745d7ae147f086c1b80b1578c29c" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 44921 } } peers { permanent_uuid: "616c869f814e47a4b9ed79fdd4cb090e" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 34083 } }
I20260806 03:41:59.755877 20980 ts_tablet_manager.cc:1428] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 03:41:59.757689 20826 tablet_copy_service.cc:342] P 75740c963c7b48c886f86c2693e1ca3d: Request end of tablet copy session 616c869f814e47a4b9ed79fdd4cb090e-12375a74ecf8476aa87de61ef2e8de70 received from {username='slave'} at 127.0.0.1:56106
I20260806 03:41:59.758119 20826 tablet_copy_service.cc:433] P 75740c963c7b48c886f86c2693e1ca3d: ending tablet copy session 616c869f814e47a4b9ed79fdd4cb090e-12375a74ecf8476aa87de61ef2e8de70 on tablet 12375a74ecf8476aa87de61ef2e8de70 with peer 616c869f814e47a4b9ed79fdd4cb090e
I20260806 03:41:59.879398 20647 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260806 03:41:59.888724 20806 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260806 03:41:59.892737 20724 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
e5b6c3cc1d444896945227446d577102 | 127.15.252.126:38131 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
13b8745d7ae147f086c1b80b1578c29c | 127.15.252.65:44921 | HEALTHY | <none> | 0 | 0
616c869f814e47a4b9ed79fdd4cb090e | 127.15.252.66:34083 | HEALTHY | <none> | 0 | 0
75740c963c7b48c886f86c2693e1ca3d | 127.15.252.67:43441 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet 12375a74ecf8476aa87de61ef2e8de70 of table 'test-workload' is conflicted: 1 replicas' active configs disagree with the leader master's
75740c963c7b48c886f86c2693e1ca3d (127.15.252.67:43441): RUNNING [LEADER]
13b8745d7ae147f086c1b80b1578c29c (127.15.252.65:44921): RUNNING
616c869f814e47a4b9ed79fdd4cb090e (127.15.252.66:34083): RUNNING
All reported replicas are:
A = 75740c963c7b48c886f86c2693e1ca3d
B = 13b8745d7ae147f086c1b80b1578c29c
C = 616c869f814e47a4b9ed79fdd4cb090e
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 1 | 3 | Yes
B | A* B C | 1 | 3 | Yes
C | A B C | 1 | 3 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260806 03:42:00.096349 16369 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260806 03:42:00.180759 20735 raft_consensus.cc:1203] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[1.3-1.3] Dedup: 1.3->[]
I20260806 03:42:00.576638 20724 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260806 03:42:00.581928 20806 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260806 03:42:00.582962 20647 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
e5b6c3cc1d444896945227446d577102 | 127.15.252.126:38131 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
13b8745d7ae147f086c1b80b1578c29c | 127.15.252.65:44921 | HEALTHY | <none> | 0 | 0
616c869f814e47a4b9ed79fdd4cb090e | 127.15.252.66:34083 | HEALTHY | <none> | 0 | 0
75740c963c7b48c886f86c2693e1ca3d | 127.15.252.67:43441 | HEALTHY | <none> | 1 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
------------------------------------------+-------+-----------------------+-------------------------
raft_prepare_replacement_before_eviction | false | experimental,advanced | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260806 03:42:00.806870 16369 log_verifier.cc:126] Checking tablet 12375a74ecf8476aa87de61ef2e8de70
I20260806 03:42:00.877151 16369 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 12375a74ecf8476aa87de61ef2e8de70
I20260806 03:42:00.881693 16369 tablet_server.cc:178] TabletServer@127.15.252.65:0 shutting down...
I20260806 03:42:00.915441 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:42:00.916333 16369 tablet_replica.cc:330] T 12375a74ecf8476aa87de61ef2e8de70 P 13b8745d7ae147f086c1b80b1578c29c: stopping tablet replica
I20260806 03:42:00.917004 16369 raft_consensus.cc:2229] T 12375a74ecf8476aa87de61ef2e8de70 P 13b8745d7ae147f086c1b80b1578c29c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:42:00.917488 16369 raft_consensus.cc:2258] T 12375a74ecf8476aa87de61ef2e8de70 P 13b8745d7ae147f086c1b80b1578c29c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:42:00.941912 16369 tablet_server.cc:194] TabletServer@127.15.252.65:0 shutdown complete.
I20260806 03:42:00.955152 16369 tablet_server.cc:178] TabletServer@127.15.252.66:0 shutting down...
I20260806 03:42:00.984190 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:42:00.984943 16369 tablet_replica.cc:330] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e: stopping tablet replica
I20260806 03:42:00.985576 16369 raft_consensus.cc:2229] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:42:00.986034 16369 raft_consensus.cc:2258] T 12375a74ecf8476aa87de61ef2e8de70 P 616c869f814e47a4b9ed79fdd4cb090e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:42:01.008908 16369 tablet_server.cc:194] TabletServer@127.15.252.66:0 shutdown complete.
I20260806 03:42:01.022135 16369 tablet_server.cc:178] TabletServer@127.15.252.67:0 shutting down...
I20260806 03:42:01.049345 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:42:01.050101 16369 tablet_replica.cc:330] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d: stopping tablet replica
I20260806 03:42:01.050693 16369 raft_consensus.cc:2229] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:42:01.051528 16369 raft_consensus.cc:2258] T 12375a74ecf8476aa87de61ef2e8de70 P 75740c963c7b48c886f86c2693e1ca3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:42:01.076439 16369 tablet_server.cc:194] TabletServer@127.15.252.67:0 shutdown complete.
I20260806 03:42:01.089570 16369 master.cc:537] Master@127.15.252.126:38131 shutting down...
I20260806 03:42:01.112483 16369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:42:01.113070 16369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:42:01.113430 16369 tablet_replica.cc:330] T 00000000000000000000000000000000 P e5b6c3cc1d444896945227446d577102: stopping tablet replica
I20260806 03:42:01.132505 16369 master.cc:552] Master@127.15.252.126:38131 shutdown complete.
[ OK ] FailedTabletsAreReplacedITest.OneReplica/0 (4341 ms)
[ RUN ] FailedTabletsAreReplacedITest.OneReplica/1
I20260806 03:42:01.169421 16369 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.15.252.126:42651
I20260806 03:42:01.171363 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:42:01.178530 21036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:42:01.180069 21037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:42:01.182528 16369 server_base.cc:1034] running on GCE node
W20260806 03:42:01.184983 21040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:42:01.186031 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:42:01.186362 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:42:01.186579 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987721186564 us; error 0 us; skew 500 ppm
I20260806 03:42:01.187485 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:42:01.191486 16369 webserver.cc:460] Webserver started at http://127.15.252.126:46331/ using document root <none> and password file <none>
I20260806 03:42:01.192144 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:42:01.192499 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:42:01.192972 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:42:01.194597 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/master-0-root/instance:
uuid: "70a712f60dd449ec881b667b4f80c4b6"
format_stamp: "Formatted at 2026-08-06 03:42:01 on dist-test-slave-csg5"
I20260806 03:42:01.201786 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260806 03:42:01.207376 21044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:42:01.208668 16369 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260806 03:42:01.209065 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/master-0-root
uuid: "70a712f60dd449ec881b667b4f80c4b6"
format_stamp: "Formatted at 2026-08-06 03:42:01 on dist-test-slave-csg5"
I20260806 03:42:01.209491 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:42:01.232326 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:42:01.234011 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:42:01.288522 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.126:42651
I20260806 03:42:01.288797 21097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.126:42651 every 8 connection(s)
I20260806 03:42:01.293365 21098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:42:01.309367 21098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6: Bootstrap starting.
I20260806 03:42:01.314596 21098 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6: Neither blocks nor log segments found. Creating new log.
I20260806 03:42:01.319566 21098 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6: No bootstrap required, opened a new log
I20260806 03:42:01.321703 21098 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a712f60dd449ec881b667b4f80c4b6" member_type: VOTER }
I20260806 03:42:01.322151 21098 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:42:01.322332 21098 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a712f60dd449ec881b667b4f80c4b6, State: Initialized, Role: FOLLOWER
I20260806 03:42:01.322957 21098 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a712f60dd449ec881b667b4f80c4b6" member_type: VOTER }
I20260806 03:42:01.323431 21098 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:42:01.323611 21098 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:42:01.323822 21098 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:42:01.328789 21098 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a712f60dd449ec881b667b4f80c4b6" member_type: VOTER }
I20260806 03:42:01.329310 21098 leader_election.cc:302] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70a712f60dd449ec881b667b4f80c4b6; no voters:
I20260806 03:42:01.330606 21098 leader_election.cc:288] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:42:01.331347 21101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:42:01.332873 21101 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6 [term 1 LEADER]: Becoming Leader. State: Replica: 70a712f60dd449ec881b667b4f80c4b6, State: Running, Role: LEADER
I20260806 03:42:01.333904 21098 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:42:01.333714 21101 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a712f60dd449ec881b667b4f80c4b6" member_type: VOTER }
I20260806 03:42:01.338085 21103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70a712f60dd449ec881b667b4f80c4b6. Latest consensus state: current_term: 1 leader_uuid: "70a712f60dd449ec881b667b4f80c4b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a712f60dd449ec881b667b4f80c4b6" member_type: VOTER } }
I20260806 03:42:01.339457 21103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6 [sys.catalog]: This master's current role is: LEADER
I20260806 03:42:01.339214 21102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70a712f60dd449ec881b667b4f80c4b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70a712f60dd449ec881b667b4f80c4b6" member_type: VOTER } }
I20260806 03:42:01.342116 21102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6 [sys.catalog]: This master's current role is: LEADER
I20260806 03:42:01.344393 21108 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:42:01.351460 21108 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:42:01.352571 16369 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260806 03:42:01.362159 21108 catalog_manager.cc:1347] Generated new cluster ID: e8f161b598f941108d1fcd853fa7cd01
I20260806 03:42:01.362524 21108 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:42:01.385303 21108 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:42:01.387530 21108 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:42:01.401947 21108 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6: Generated new TSK 0
I20260806 03:42:01.402819 21108 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:42:01.419951 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:42:01.428619 21119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:42:01.428710 21120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:42:01.432389 16369 server_base.cc:1034] running on GCE node
W20260806 03:42:01.432741 21122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:42:01.433928 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:42:01.434226 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:42:01.434556 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987721434529 us; error 0 us; skew 500 ppm
I20260806 03:42:01.435237 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:42:01.442178 16369 webserver.cc:460] Webserver started at http://127.15.252.65:40949/ using document root <none> and password file <none>
I20260806 03:42:01.442870 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:42:01.443086 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:42:01.443336 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:42:01.444720 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-0-root/instance:
uuid: "7ab58ca9b9cb4287bd57c1b81267404a"
format_stamp: "Formatted at 2026-08-06 03:42:01 on dist-test-slave-csg5"
I20260806 03:42:01.450485 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260806 03:42:01.455470 21127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:42:01.456501 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260806 03:42:01.456853 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-0-root
uuid: "7ab58ca9b9cb4287bd57c1b81267404a"
format_stamp: "Formatted at 2026-08-06 03:42:01 on dist-test-slave-csg5"
I20260806 03:42:01.457219 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:42:01.477559 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:42:01.479388 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:42:01.481137 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:42:01.483899 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:42:01.484149 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:42:01.484431 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:42:01.484612 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:42:01.535707 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.65:41661
I20260806 03:42:01.535808 21189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.65:41661 every 8 connection(s)
I20260806 03:42:01.540803 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:42:01.549962 21195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:42:01.552961 21196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:42:01.554011 16369 server_base.cc:1034] running on GCE node
W20260806 03:42:01.554175 21198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:42:01.555518 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:42:01.555794 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:42:01.556003 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987721555985 us; error 0 us; skew 500 ppm
I20260806 03:42:01.556737 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:42:01.559139 21190 heartbeater.cc:347] Connected to a master server at 127.15.252.126:42651
I20260806 03:42:01.559796 21190 heartbeater.cc:464] Registering TS with master...
I20260806 03:42:01.560849 21190 heartbeater.cc:511] Master 127.15.252.126:42651 requested a full tablet report, sending...
I20260806 03:42:01.560956 16369 webserver.cc:460] Webserver started at http://127.15.252.66:46299/ using document root <none> and password file <none>
I20260806 03:42:01.561694 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:42:01.561985 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:42:01.562327 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:42:01.563812 21062 ts_manager.cc:194] Registered new tserver with Master: 7ab58ca9b9cb4287bd57c1b81267404a (127.15.252.65:41661)
I20260806 03:42:01.564067 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-1-root/instance:
uuid: "3387779120e5434481c8f4ad83c9405b"
format_stamp: "Formatted at 2026-08-06 03:42:01 on dist-test-slave-csg5"
I20260806 03:42:01.567276 21062 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39964
I20260806 03:42:01.570997 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.003s
I20260806 03:42:01.575412 21203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:42:01.576536 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 03:42:01.576898 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-1-root
uuid: "3387779120e5434481c8f4ad83c9405b"
format_stamp: "Formatted at 2026-08-06 03:42:01 on dist-test-slave-csg5"
I20260806 03:42:01.577191 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:42:01.602056 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:42:01.603493 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:42:01.605332 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:42:01.607863 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:42:01.608088 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:42:01.608333 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:42:01.608495 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:42:01.672466 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.66:37715
I20260806 03:42:01.672492 21267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.66:37715 every 8 connection(s)
I20260806 03:42:01.682787 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:42:01.693764 21272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:42:01.696298 21271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:42:01.698565 21268 heartbeater.cc:347] Connected to a master server at 127.15.252.126:42651
I20260806 03:42:01.699115 21268 heartbeater.cc:464] Registering TS with master...
I20260806 03:42:01.700107 21268 heartbeater.cc:511] Master 127.15.252.126:42651 requested a full tablet report, sending...
I20260806 03:42:01.703194 21062 ts_manager.cc:194] Registered new tserver with Master: 3387779120e5434481c8f4ad83c9405b (127.15.252.66:37715)
W20260806 03:42:01.703616 21274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:42:01.705404 21062 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39976
I20260806 03:42:01.708748 16369 server_base.cc:1034] running on GCE node
I20260806 03:42:01.710196 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:42:01.710536 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:42:01.710814 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987721710796 us; error 0 us; skew 500 ppm
I20260806 03:42:01.711678 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:42:01.715441 16369 webserver.cc:460] Webserver started at http://127.15.252.67:34289/ using document root <none> and password file <none>
I20260806 03:42:01.716205 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:42:01.716470 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:42:01.716816 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:42:01.718523 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-2-root/instance:
uuid: "fb8eb93b8db6498786dd1e68b905f221"
format_stamp: "Formatted at 2026-08-06 03:42:01 on dist-test-slave-csg5"
I20260806 03:42:01.724777 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260806 03:42:01.729072 21279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:42:01.730365 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 03:42:01.730894 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-2-root
uuid: "fb8eb93b8db6498786dd1e68b905f221"
format_stamp: "Formatted at 2026-08-06 03:42:01 on dist-test-slave-csg5"
I20260806 03:42:01.731210 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:42:01.742446 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:42:01.743907 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:42:01.745867 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:42:01.748755 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:42:01.749068 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:42:01.749356 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:42:01.749550 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:42:01.805233 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.67:35555
I20260806 03:42:01.805437 21343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.67:35555 every 8 connection(s)
I20260806 03:42:01.810757 16369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:42:01.818095 21347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:42:01.820719 21348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:42:01.823783 21350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:42:01.824730 16369 server_base.cc:1034] running on GCE node
I20260806 03:42:01.825801 16369 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260806 03:42:01.826078 16369 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260806 03:42:01.826067 21344 heartbeater.cc:347] Connected to a master server at 127.15.252.126:42651
I20260806 03:42:01.826534 16369 hybrid_clock.cc:648] HybridClock initialized: now 1785987721826512 us; error 0 us; skew 500 ppm
I20260806 03:42:01.826789 21344 heartbeater.cc:464] Registering TS with master...
I20260806 03:42:01.827467 16369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260806 03:42:01.827728 21344 heartbeater.cc:511] Master 127.15.252.126:42651 requested a full tablet report, sending...
I20260806 03:42:01.830340 21062 ts_manager.cc:194] Registered new tserver with Master: fb8eb93b8db6498786dd1e68b905f221 (127.15.252.67:35555)
I20260806 03:42:01.831267 16369 webserver.cc:460] Webserver started at http://127.15.252.68:45035/ using document root <none> and password file <none>
I20260806 03:42:01.831852 16369 fs_manager.cc:359] Metadata directory not provided
I20260806 03:42:01.832068 16369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:42:01.832399 16369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:42:01.832623 21062 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39992
I20260806 03:42:01.834079 16369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-3-root/instance:
uuid: "459893947b7f48bf8b587185624a4184"
format_stamp: "Formatted at 2026-08-06 03:42:01 on dist-test-slave-csg5"
I20260806 03:42:01.839859 16369 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260806 03:42:01.843576 21355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:42:01.844624 16369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260806 03:42:01.844972 16369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-3-root
uuid: "459893947b7f48bf8b587185624a4184"
format_stamp: "Formatted at 2026-08-06 03:42:01 on dist-test-slave-csg5"
I20260806 03:42:01.845284 16369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:42:01.867704 16369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:42:01.869138 16369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:42:01.870640 16369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:42:01.873405 16369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:42:01.873629 16369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:42:01.873965 16369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:42:01.874119 16369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:42:01.922771 16369 rpc_server.cc:307] RPC server started. Bound to: 127.15.252.68:33161
I20260806 03:42:01.922916 21424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.252.68:33161 every 8 connection(s)
I20260806 03:42:01.941722 21425 heartbeater.cc:347] Connected to a master server at 127.15.252.126:42651
I20260806 03:42:01.942222 21425 heartbeater.cc:464] Registering TS with master...
I20260806 03:42:01.943142 21425 heartbeater.cc:511] Master 127.15.252.126:42651 requested a full tablet report, sending...
I20260806 03:42:01.945492 21062 ts_manager.cc:194] Registered new tserver with Master: 459893947b7f48bf8b587185624a4184 (127.15.252.68:33161)
I20260806 03:42:01.946153 16369 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.02022574s
I20260806 03:42:01.947945 21062 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40004
I20260806 03:42:01.973299 16369 test_util.cc:268] Using random seed: -1413624295
I20260806 03:42:02.000442 21062 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40022:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 03:42:02.051930 21155 tablet_service.cc:1467] Processing CreateTablet for tablet 738bd22dcffc47f0b2fcf51fc8773ff3 (DEFAULT_TABLE table=test-workload [id=cfc0caf9c6ca4a7a8f97768e56c4871e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:42:02.053318 21155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 738bd22dcffc47f0b2fcf51fc8773ff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:42:02.057566 21308 tablet_service.cc:1467] Processing CreateTablet for tablet 738bd22dcffc47f0b2fcf51fc8773ff3 (DEFAULT_TABLE table=test-workload [id=cfc0caf9c6ca4a7a8f97768e56c4871e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:42:02.058977 21308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 738bd22dcffc47f0b2fcf51fc8773ff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:42:02.061038 21232 tablet_service.cc:1467] Processing CreateTablet for tablet 738bd22dcffc47f0b2fcf51fc8773ff3 (DEFAULT_TABLE table=test-workload [id=cfc0caf9c6ca4a7a8f97768e56c4871e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:42:02.062294 21232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 738bd22dcffc47f0b2fcf51fc8773ff3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:42:02.074214 21448 tablet_bootstrap.cc:492] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a: Bootstrap starting.
I20260806 03:42:02.078323 21449 tablet_bootstrap.cc:492] T 738bd22dcffc47f0b2fcf51fc8773ff3 P fb8eb93b8db6498786dd1e68b905f221: Bootstrap starting.
I20260806 03:42:02.082190 21448 tablet_bootstrap.cc:654] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a: Neither blocks nor log segments found. Creating new log.
I20260806 03:42:02.085161 21449 tablet_bootstrap.cc:654] T 738bd22dcffc47f0b2fcf51fc8773ff3 P fb8eb93b8db6498786dd1e68b905f221: Neither blocks nor log segments found. Creating new log.
I20260806 03:42:02.086807 21450 tablet_bootstrap.cc:492] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 3387779120e5434481c8f4ad83c9405b: Bootstrap starting.
I20260806 03:42:02.090052 21448 tablet_bootstrap.cc:492] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a: No bootstrap required, opened a new log
I20260806 03:42:02.090482 21448 ts_tablet_manager.cc:1397] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.007s
I20260806 03:42:02.091337 21449 tablet_bootstrap.cc:492] T 738bd22dcffc47f0b2fcf51fc8773ff3 P fb8eb93b8db6498786dd1e68b905f221: No bootstrap required, opened a new log
I20260806 03:42:02.091886 21449 ts_tablet_manager.cc:1397] T 738bd22dcffc47f0b2fcf51fc8773ff3 P fb8eb93b8db6498786dd1e68b905f221: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.006s
I20260806 03:42:02.092803 21448 raft_consensus.cc:348] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 41661 } } peers { permanent_uuid: "fb8eb93b8db6498786dd1e68b905f221" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35555 } } peers { permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 } }
I20260806 03:42:02.093581 21448 raft_consensus.cc:374] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:42:02.093930 21448 raft_consensus.cc:729] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ab58ca9b9cb4287bd57c1b81267404a, State: Initialized, Role: FOLLOWER
I20260806 03:42:02.094233 21450 tablet_bootstrap.cc:654] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 3387779120e5434481c8f4ad83c9405b: Neither blocks nor log segments found. Creating new log.
I20260806 03:42:02.094759 21448 consensus_queue.cc:260] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 41661 } } peers { permanent_uuid: "fb8eb93b8db6498786dd1e68b905f221" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35555 } } peers { permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 } }
I20260806 03:42:02.095134 21449 raft_consensus.cc:348] T 738bd22dcffc47f0b2fcf51fc8773ff3 P fb8eb93b8db6498786dd1e68b905f221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 41661 } } peers { permanent_uuid: "fb8eb93b8db6498786dd1e68b905f221" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35555 } } peers { permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 } }
I20260806 03:42:02.096155 21449 raft_consensus.cc:374] T 738bd22dcffc47f0b2fcf51fc8773ff3 P fb8eb93b8db6498786dd1e68b905f221 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:42:02.096580 21449 raft_consensus.cc:729] T 738bd22dcffc47f0b2fcf51fc8773ff3 P fb8eb93b8db6498786dd1e68b905f221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb8eb93b8db6498786dd1e68b905f221, State: Initialized, Role: FOLLOWER
I20260806 03:42:02.097570 21190 heartbeater.cc:503] Master 127.15.252.126:42651 was elected leader, sending a full tablet report...
I20260806 03:42:02.098699 21448 ts_tablet_manager.cc:1428] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
I20260806 03:42:02.097923 21449 consensus_queue.cc:260] T 738bd22dcffc47f0b2fcf51fc8773ff3 P fb8eb93b8db6498786dd1e68b905f221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 41661 } } peers { permanent_uuid: "fb8eb93b8db6498786dd1e68b905f221" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35555 } } peers { permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 } }
I20260806 03:42:02.105382 21344 heartbeater.cc:503] Master 127.15.252.126:42651 was elected leader, sending a full tablet report...
I20260806 03:42:02.106081 21449 ts_tablet_manager.cc:1428] T 738bd22dcffc47f0b2fcf51fc8773ff3 P fb8eb93b8db6498786dd1e68b905f221: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260806 03:42:02.107592 21450 tablet_bootstrap.cc:492] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 3387779120e5434481c8f4ad83c9405b: No bootstrap required, opened a new log
I20260806 03:42:02.108258 21450 ts_tablet_manager.cc:1397] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 3387779120e5434481c8f4ad83c9405b: Time spent bootstrapping tablet: real 0.022s user 0.013s sys 0.008s
I20260806 03:42:02.111630 21450 raft_consensus.cc:348] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 3387779120e5434481c8f4ad83c9405b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 41661 } } peers { permanent_uuid: "fb8eb93b8db6498786dd1e68b905f221" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35555 } } peers { permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 } }
I20260806 03:42:02.112278 21450 raft_consensus.cc:374] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 3387779120e5434481c8f4ad83c9405b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:42:02.112557 21450 raft_consensus.cc:729] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 3387779120e5434481c8f4ad83c9405b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3387779120e5434481c8f4ad83c9405b, State: Initialized, Role: FOLLOWER
I20260806 03:42:02.113240 21450 consensus_queue.cc:260] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 3387779120e5434481c8f4ad83c9405b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 41661 } } peers { permanent_uuid: "fb8eb93b8db6498786dd1e68b905f221" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35555 } } peers { permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 } }
I20260806 03:42:02.115700 21268 heartbeater.cc:503] Master 127.15.252.126:42651 was elected leader, sending a full tablet report...
I20260806 03:42:02.116456 21450 ts_tablet_manager.cc:1428] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 3387779120e5434481c8f4ad83c9405b: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260806 03:42:02.181640 21453 raft_consensus.cc:482] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:42:02.182202 21453 raft_consensus.cc:504] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 41661 } } peers { permanent_uuid: "fb8eb93b8db6498786dd1e68b905f221" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35555 } } peers { permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 } }
I20260806 03:42:02.184314 21453 leader_election.cc:288] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb8eb93b8db6498786dd1e68b905f221 (127.15.252.67:35555), 3387779120e5434481c8f4ad83c9405b (127.15.252.66:37715)
I20260806 03:42:02.199735 21242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "738bd22dcffc47f0b2fcf51fc8773ff3" candidate_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3387779120e5434481c8f4ad83c9405b" is_pre_election: true
I20260806 03:42:02.200565 21242 raft_consensus.cc:2454] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 3387779120e5434481c8f4ad83c9405b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ab58ca9b9cb4287bd57c1b81267404a in term 0.
I20260806 03:42:02.200600 21318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "738bd22dcffc47f0b2fcf51fc8773ff3" candidate_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb8eb93b8db6498786dd1e68b905f221" is_pre_election: true
I20260806 03:42:02.201519 21318 raft_consensus.cc:2454] T 738bd22dcffc47f0b2fcf51fc8773ff3 P fb8eb93b8db6498786dd1e68b905f221 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ab58ca9b9cb4287bd57c1b81267404a in term 0.
I20260806 03:42:02.201716 21131 leader_election.cc:302] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3387779120e5434481c8f4ad83c9405b, 7ab58ca9b9cb4287bd57c1b81267404a; no voters:
I20260806 03:42:02.202605 21453 raft_consensus.cc:2781] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:42:02.203225 21453 raft_consensus.cc:482] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:42:02.203560 21453 raft_consensus.cc:3037] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:42:02.210301 21453 raft_consensus.cc:504] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 41661 } } peers { permanent_uuid: "fb8eb93b8db6498786dd1e68b905f221" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35555 } } peers { permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 } }
I20260806 03:42:02.212529 21453 leader_election.cc:288] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [CANDIDATE]: Term 1 election: Requested vote from peers fb8eb93b8db6498786dd1e68b905f221 (127.15.252.67:35555), 3387779120e5434481c8f4ad83c9405b (127.15.252.66:37715)
I20260806 03:42:02.213400 21318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "738bd22dcffc47f0b2fcf51fc8773ff3" candidate_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb8eb93b8db6498786dd1e68b905f221"
I20260806 03:42:02.213582 21242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "738bd22dcffc47f0b2fcf51fc8773ff3" candidate_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3387779120e5434481c8f4ad83c9405b"
I20260806 03:42:02.214090 21318 raft_consensus.cc:3037] T 738bd22dcffc47f0b2fcf51fc8773ff3 P fb8eb93b8db6498786dd1e68b905f221 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:42:02.214125 21242 raft_consensus.cc:3037] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 3387779120e5434481c8f4ad83c9405b [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:42:02.221746 21242 raft_consensus.cc:2454] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 3387779120e5434481c8f4ad83c9405b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ab58ca9b9cb4287bd57c1b81267404a in term 1.
I20260806 03:42:02.221746 21318 raft_consensus.cc:2454] T 738bd22dcffc47f0b2fcf51fc8773ff3 P fb8eb93b8db6498786dd1e68b905f221 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ab58ca9b9cb4287bd57c1b81267404a in term 1.
I20260806 03:42:02.223503 21131 leader_election.cc:302] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3387779120e5434481c8f4ad83c9405b, 7ab58ca9b9cb4287bd57c1b81267404a; no voters:
I20260806 03:42:02.224431 21453 raft_consensus.cc:2781] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:42:02.225521 21453 raft_consensus.cc:686] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [term 1 LEADER]: Becoming Leader. State: Replica: 7ab58ca9b9cb4287bd57c1b81267404a, State: Running, Role: LEADER
I20260806 03:42:02.226248 21453 consensus_queue.cc:237] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 41661 } } peers { permanent_uuid: "fb8eb93b8db6498786dd1e68b905f221" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35555 } } peers { permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 } }
I20260806 03:42:02.233953 21061 catalog_manager.cc:5495] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7ab58ca9b9cb4287bd57c1b81267404a (127.15.252.65). New cstate: current_term: 1 leader_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 41661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb8eb93b8db6498786dd1e68b905f221" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35555 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 } health_report { overall_health: UNKNOWN } } }
I20260806 03:42:02.520743 21232 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260806 03:42:02.534492 21387 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260806 03:42:02.542186 21155 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260806 03:42:02.543534 21308 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260806 03:42:02.694519 21453 consensus_queue.cc:1035] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260806 03:42:02.740211 21453 consensus_queue.cc:1035] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb8eb93b8db6498786dd1e68b905f221" member_type: VOTER last_known_addr { host: "127.15.252.67" port: 35555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
70a712f60dd449ec881b667b4f80c4b6 | 127.15.252.126:42651 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
3387779120e5434481c8f4ad83c9405b | 127.15.252.66:37715 | HEALTHY | <none> | 0 | 0
459893947b7f48bf8b587185624a4184 | 127.15.252.68:33161 | HEALTHY | <none> | 0 | 0
7ab58ca9b9cb4287bd57c1b81267404a | 127.15.252.65:41661 | HEALTHY | <none> | 1 | 0
fb8eb93b8db6498786dd1e68b905f221 | 127.15.252.67:35555 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
Tablet 738bd22dcffc47f0b2fcf51fc8773ff3 of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
7ab58ca9b9cb4287bd57c1b81267404a (127.15.252.65:41661): RUNNING [LEADER]
fb8eb93b8db6498786dd1e68b905f221 (127.15.252.67:35555): RUNNING
3387779120e5434481c8f4ad83c9405b (127.15.252.66:37715): RUNNING
All reported replicas are:
A = 7ab58ca9b9cb4287bd57c1b81267404a
B = fb8eb93b8db6498786dd1e68b905f221
C = 3387779120e5434481c8f4ad83c9405b
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A* B C | | | Yes
A | A* B C | 1 | -1 | Yes
B | A B C | 1 | -1 | Yes
C | A B C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+---------------------+---------------
Small | 459893947b7f48bf8b587185624a4184 | 127.15.252.68:33161 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20260806 03:42:02.857537 16369 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20260806 03:42:02.952128 21425 heartbeater.cc:503] Master 127.15.252.126:42651 was elected leader, sending a full tablet report...
I20260806 03:42:03.275640 21387 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260806 03:42:03.284116 21155 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260806 03:42:03.287593 21232 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260806 03:42:03.291719 21308 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
70a712f60dd449ec881b667b4f80c4b6 | 127.15.252.126:42651 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
3387779120e5434481c8f4ad83c9405b | 127.15.252.66:37715 | HEALTHY | <none> | 0 | 0
459893947b7f48bf8b587185624a4184 | 127.15.252.68:33161 | HEALTHY | <none> | 0 | 0
7ab58ca9b9cb4287bd57c1b81267404a | 127.15.252.65:41661 | HEALTHY | <none> | 1 | 0
fb8eb93b8db6498786dd1e68b905f221 | 127.15.252.67:35555 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+---------------------+---------------
Small | 459893947b7f48bf8b587185624a4184 | 127.15.252.68:33161 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260806 03:42:03.543378 16369 log_verifier.cc:126] Checking tablet 738bd22dcffc47f0b2fcf51fc8773ff3
I20260806 03:42:03.615432 16369 log_verifier.cc:177] Verified matching terms for 1 ops in tablet 738bd22dcffc47f0b2fcf51fc8773ff3
I20260806 03:42:03.616003 16369 tablet_replica.cc:330] T 738bd22dcffc47f0b2fcf51fc8773ff3 P fb8eb93b8db6498786dd1e68b905f221: stopping tablet replica
I20260806 03:42:03.616580 16369 raft_consensus.cc:2229] T 738bd22dcffc47f0b2fcf51fc8773ff3 P fb8eb93b8db6498786dd1e68b905f221 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:42:03.617066 16369 raft_consensus.cc:2258] T 738bd22dcffc47f0b2fcf51fc8773ff3 P fb8eb93b8db6498786dd1e68b905f221 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:42:03.683189 21128 consensus_queue.cc:579] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [LEADER]: The tablet replica hosted on peer fb8eb93b8db6498786dd1e68b905f221 has failed
W20260806 03:42:03.683756 21128 consensus_peers.cc:487] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a -> Peer fb8eb93b8db6498786dd1e68b905f221 (127.15.252.67:35555): Couldn't send request to peer fb8eb93b8db6498786dd1e68b905f221. Error code: TABLET_FAILED (20). Status: Illegal state: Tablet not RUNNING: FAILED: IO error: INJECTED ERROR: tablet failed. This is attempt 1: this message will repeat every 5th retry.
I20260806 03:42:03.692081 21165 consensus_queue.cc:237] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 41661 } } peers { permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 } }
I20260806 03:42:03.695950 21241 raft_consensus.cc:1261] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 3387779120e5434481c8f4ad83c9405b [term 1 FOLLOWER]: Refusing update from remote peer 7ab58ca9b9cb4287bd57c1b81267404a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:42:03.697540 21531 consensus_queue.cc:1035] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260806 03:42:03.704319 21532 raft_consensus.cc:2932] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fb8eb93b8db6498786dd1e68b905f221 (127.15.252.67) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 41661 } } peers { permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 } } }
I20260806 03:42:03.706281 21241 raft_consensus.cc:2932] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 3387779120e5434481c8f4ad83c9405b [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER fb8eb93b8db6498786dd1e68b905f221 (127.15.252.67) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 41661 } } peers { permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 } } }
I20260806 03:42:03.713104 21047 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 738bd22dcffc47f0b2fcf51fc8773ff3 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260806 03:42:03.717548 21059 catalog_manager.cc:5495] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a reported cstate change: config changed from index -1 to 2, VOTER fb8eb93b8db6498786dd1e68b905f221 (127.15.252.67) evicted. New cstate: current_term: 1 leader_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 41661 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 } health_report { overall_health: HEALTHY } } }
I20260806 03:42:03.734712 21308 tablet_service.cc:1514] Processing DeleteTablet for tablet 738bd22dcffc47f0b2fcf51fc8773ff3 with delete_type TABLET_DATA_TOMBSTONED (TS fb8eb93b8db6498786dd1e68b905f221 not found in new config with opid_index 2) from {username='slave'} at 127.0.0.1:38806
I20260806 03:42:03.737936 21545 ts_tablet_manager.cc:1905] T 738bd22dcffc47f0b2fcf51fc8773ff3 P fb8eb93b8db6498786dd1e68b905f221: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260806 03:42:03.739732 21165 consensus_queue.cc:237] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 41661 } } peers { permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 } } peers { permanent_uuid: "459893947b7f48bf8b587185624a4184" member_type: NON_VOTER last_known_addr { host: "127.15.252.68" port: 33161 } attrs { promote: true } }
I20260806 03:42:03.749048 21241 raft_consensus.cc:1261] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 3387779120e5434481c8f4ad83c9405b [term 1 FOLLOWER]: Refusing update from remote peer 7ab58ca9b9cb4287bd57c1b81267404a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260806 03:42:03.751024 21532 consensus_queue.cc:1035] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260806 03:42:03.759392 21459 raft_consensus.cc:2932] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 459893947b7f48bf8b587185624a4184 (127.15.252.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 41661 } } peers { permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 } } peers { permanent_uuid: "459893947b7f48bf8b587185624a4184" member_type: NON_VOTER last_known_addr { host: "127.15.252.68" port: 33161 } attrs { promote: true } } }
I20260806 03:42:03.760924 21545 ts_tablet_manager.cc:1918] T 738bd22dcffc47f0b2fcf51fc8773ff3 P fb8eb93b8db6498786dd1e68b905f221: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.1
I20260806 03:42:03.761592 21545 log.cc:1196] T 738bd22dcffc47f0b2fcf51fc8773ff3 P fb8eb93b8db6498786dd1e68b905f221: Deleting WAL directory at /tmp/dist-test-taskID12CW/test-tmp/ts_tablet_manager-itest.0.FailedTabletsAreReplacedITest.OneReplica_1.1785987678583700-16369-0/minicluster-data/ts-2-root/wals/738bd22dcffc47f0b2fcf51fc8773ff3
I20260806 03:42:03.761302 21241 raft_consensus.cc:2932] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 3387779120e5434481c8f4ad83c9405b [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER 459893947b7f48bf8b587185624a4184 (127.15.252.68) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 41661 } } peers { permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 } } peers { permanent_uuid: "459893947b7f48bf8b587185624a4184" member_type: NON_VOTER last_known_addr { host: "127.15.252.68" port: 33161 } attrs { promote: true } } }
I20260806 03:42:03.764330 21045 catalog_manager.cc:4841] TS fb8eb93b8db6498786dd1e68b905f221 (127.15.252.67:35555): tablet 738bd22dcffc47f0b2fcf51fc8773ff3 (table test-workload [id=cfc0caf9c6ca4a7a8f97768e56c4871e]) successfully deleted
I20260806 03:42:03.774528 21047 catalog_manager.cc:5008] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 738bd22dcffc47f0b2fcf51fc8773ff3 with cas_config_opid_index 2: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20260806 03:42:03.776973 21059 catalog_manager.cc:5495] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 3387779120e5434481c8f4ad83c9405b reported cstate change: config changed from index 2 to 3, NON_VOTER 459893947b7f48bf8b587185624a4184 (127.15.252.68) added. New cstate: current_term: 1 leader_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "7ab58ca9b9cb4287bd57c1b81267404a" member_type: VOTER last_known_addr { host: "127.15.252.65" port: 41661 } } peers { permanent_uuid: "3387779120e5434481c8f4ad83c9405b" member_type: VOTER last_known_addr { host: "127.15.252.66" port: 37715 } } peers { permanent_uuid: "459893947b7f48bf8b587185624a4184" member_type: NON_VOTER last_known_addr { host: "127.15.252.68" port: 33161 } attrs { promote: true } } }
W20260806 03:42:03.784837 21128 consensus_peers.cc:487] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a -> Peer 459893947b7f48bf8b587185624a4184 (127.15.252.68:33161): Couldn't send request to peer 459893947b7f48bf8b587185624a4184. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 738bd22dcffc47f0b2fcf51fc8773ff3. This is attempt 1: this message will repeat every 5th retry.
I20260806 03:42:03.898717 21232 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260806 03:42:03.911180 21387 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
I20260806 03:42:03.913029 21155 tablet_service.cc:1429] Tablet server has 1 leaders and 0 scanners
I20260806 03:42:03.924777 21308 tablet_service.cc:1429] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
70a712f60dd449ec881b667b4f80c4b6 | 127.15.252.126:42651 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+---------------------+---------+----------+----------------+-----------------
3387779120e5434481c8f4ad83c9405b | 127.15.252.66:37715 | HEALTHY | <none> | 0 | 0
459893947b7f48bf8b587185624a4184 | 127.15.252.68:33161 | HEALTHY | <none> | 0 | 0
7ab58ca9b9cb4287bd57c1b81267404a | 127.15.252.65:41661 | HEALTHY | <none> | 1 | 0
fb8eb93b8db6498786dd1e68b905f221 | 127.15.252.67:35555 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 4
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+--------+-------------------------
rpc_server_allow_ephemeral_ports | true | unsafe | all 4 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 4 server(s) checked
time_source | system_unsync | all 4 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.18.0-SNAPSHOT | all 5 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 0
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Tablet Replica Count Outliers
Type | UUID | Host | Replica Count
-------+----------------------------------+---------------------+---------------
Small | fb8eb93b8db6498786dd1e68b905f221 | 127.15.252.67:35555 | 0
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 4
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20260806 03:42:04.165612 16369 log_verifier.cc:126] Checking tablet 738bd22dcffc47f0b2fcf51fc8773ff3
I20260806 03:42:04.207536 16369 log_verifier.cc:177] Verified matching terms for 3 ops in tablet 738bd22dcffc47f0b2fcf51fc8773ff3
I20260806 03:42:04.210006 16369 tablet_server.cc:178] TabletServer@127.15.252.65:0 shutting down...
I20260806 03:42:04.228988 21570 ts_tablet_manager.cc:927] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 459893947b7f48bf8b587185624a4184: Initiating tablet copy from peer 7ab58ca9b9cb4287bd57c1b81267404a (127.15.252.65:41661)
I20260806 03:42:04.233275 21570 tablet_copy_client.cc:316] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 459893947b7f48bf8b587185624a4184: tablet copy: Beginning tablet copy session from remote peer at address 127.15.252.65:41661
I20260806 03:42:04.245071 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:42:04.245923 16369 tablet_replica.cc:330] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a: stopping tablet replica
I20260806 03:42:04.246687 16369 raft_consensus.cc:2229] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:42:04.247609 16369 raft_consensus.cc:2258] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 7ab58ca9b9cb4287bd57c1b81267404a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 03:42:04.254786 21128 proxy.cc:239] Call had error, refreshing address and retrying: Service unavailable: reactor is shutting down (error 108) [suppressed 38 similar messages]
I20260806 03:42:04.262292 16369 tablet_server.cc:194] TabletServer@127.15.252.65:0 shutdown complete.
W20260806 03:42:04.264047 21359 connection.cc:441] server connection from 127.0.0.1:47458 torn down before Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:47458 (request call id 1) could send its response
I20260806 03:42:04.273866 16369 tablet_server.cc:178] TabletServer@127.15.252.66:0 shutting down...
I20260806 03:42:04.295986 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:42:04.296628 16369 tablet_replica.cc:330] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 3387779120e5434481c8f4ad83c9405b: stopping tablet replica
I20260806 03:42:04.297195 16369 raft_consensus.cc:2229] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 3387779120e5434481c8f4ad83c9405b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:42:04.297611 16369 raft_consensus.cc:2258] T 738bd22dcffc47f0b2fcf51fc8773ff3 P 3387779120e5434481c8f4ad83c9405b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:42:04.318229 16369 tablet_server.cc:194] TabletServer@127.15.252.66:0 shutdown complete.
I20260806 03:42:04.329416 16369 tablet_server.cc:178] TabletServer@127.15.252.67:0 shutting down...
I20260806 03:42:04.362614 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:42:04.381139 16369 tablet_server.cc:194] TabletServer@127.15.252.67:0 shutdown complete.
I20260806 03:42:04.391767 16369 tablet_server.cc:178] TabletServer@127.15.252.68:0 shutting down...
I20260806 03:42:04.413852 16369 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260806 03:42:04.430860 16369 tablet_server.cc:194] TabletServer@127.15.252.68:0 shutdown complete.
I20260806 03:42:04.440598 16369 master.cc:537] Master@127.15.252.126:42651 shutting down...
I20260806 03:42:04.459578 16369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:42:04.460134 16369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:42:04.460435 16369 tablet_replica.cc:330] T 00000000000000000000000000000000 P 70a712f60dd449ec881b667b4f80c4b6: stopping tablet replica
I20260806 03:42:04.480093 16369 master.cc:552] Master@127.15.252.126:42651 shutdown complete.
[ OK ] FailedTabletsAreReplacedITest.OneReplica/1 (3339 ms)
[----------] 2 tests from FailedTabletsAreReplacedITest (7681 ms total)
[----------] Global test environment tear-down
[==========] 13 tests from 4 test suites ran. (45703 ms total)
[ PASSED ] 11 tests.
[ SKIPPED ] 1 test, listed below:
[ SKIPPED ] LeadershipChangeReportingTest.TestReportStatsDuringLeadershipChange
[ FAILED ] 1 test, listed below:
[ FAILED ] TsTabletManagerITest.TestTableStats
1 FAILED TEST
I20260806 03:42:04.514569 16369 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/client/meta_cache.cc:788: suppressed but not reported on 3 messages since previous log ~8 seconds ago
I20260806 03:42:04.514951 16369 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tserver/heartbeater.cc:644: suppressed but not reported on 155 messages since previous log ~36 seconds ago
I20260806 03:42:04.515223 16369 logging.cc:424] LogThrottler /data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/tablet/tablet.cc:2369: suppressed but not reported on 9 messages since previous log ~12 seconds ago