[==========] Running 22 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 22 tests from RemoteKsckTest
[ RUN ] RemoteKsckTest.TestClusterOk
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20250904 19:57:49.578131 10338 test_util.cc:276] Using random seed: 583359257
I20250904 19:57:49.581676 10338 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.24.190:44751,127.10.24.189:46147,127.10.24.188:37175
I20250904 19:57:49.583158 10338 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250904 19:57:49.584165 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:57:49.599253 10344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:57:49.599299 10345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:49.601248 10338 server_base.cc:1047] running on GCE node
W20250904 19:57:49.600180 10347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:50.684752 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:57:50.684983 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:57:50.685169 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015870685148 us; error 0 us; skew 500 ppm
I20250904 19:57:50.685988 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:57:50.691694 10338 webserver.cc:480] Webserver started at http://127.10.24.190:34169/ using document root <none> and password file <none>
I20250904 19:57:50.692503 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:57:50.692701 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:57:50.693133 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:57:50.698743 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/master-0-root/instance:
uuid: "da0e75bb907941f8946c895ced46a241"
format_stamp: "Formatted at 2025-09-04 19:57:50 on dist-test-slave-f3xj"
I20250904 19:57:50.706136 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20250904 19:57:50.711020 10353 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:50.712069 10338 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20250904 19:57:50.712380 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/master-0-root
uuid: "da0e75bb907941f8946c895ced46a241"
format_stamp: "Formatted at 2025-09-04 19:57:50 on dist-test-slave-f3xj"
I20250904 19:57:50.712704 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:57:50.761688 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:57:50.763309 10338 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250904 19:57:50.763748 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:57:50.830885 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.190:44751
I20250904 19:57:50.830950 10404 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.190:44751 every 8 connection(s)
I20250904 19:57:50.836596 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:57:50.836796 10405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20250904 19:57:50.842614 10407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:57:50.844848 10408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:57:50.846841 10410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:50.847066 10338 server_base.cc:1047] running on GCE node
I20250904 19:57:50.847934 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:57:50.848201 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:57:50.848402 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015870848383 us; error 0 us; skew 500 ppm
I20250904 19:57:50.849148 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:57:50.853008 10338 webserver.cc:480] Webserver started at http://127.10.24.189:35261/ using document root <none> and password file <none>
I20250904 19:57:50.853689 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:57:50.853928 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:57:50.854269 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:57:50.855650 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/master-1-root/instance:
uuid: "650c62ea71eb46aaa624a6aadceb250c"
format_stamp: "Formatted at 2025-09-04 19:57:50 on dist-test-slave-f3xj"
I20250904 19:57:50.859174 10405 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:57:50.861903 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20250904 19:57:50.867384 10417 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:50.868366 10338 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20250904 19:57:50.868710 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/master-1-root
uuid: "650c62ea71eb46aaa624a6aadceb250c"
format_stamp: "Formatted at 2025-09-04 19:57:50 on dist-test-slave-f3xj"
I20250904 19:57:50.869076 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:57:50.881810 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:57:50.883042 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:57:50.888309 10405 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:57:50.890668 10357 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.24.189:46147: connect: Connection refused (error 111)
W20250904 19:57:50.894767 10405 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.189:46147: Network error: Client connection negotiation failed: client connection to 127.10.24.189:46147: connect: Connection refused (error 111)
I20250904 19:57:50.942237 10405 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } attempt: 1
W20250904 19:57:50.946648 10405 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.189:46147: Network error: Client connection negotiation failed: client connection to 127.10.24.189:46147: connect: Connection refused (error 111)
I20250904 19:57:50.949460 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.189:46147
I20250904 19:57:50.949637 10469 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.189:46147 every 8 connection(s)
I20250904 19:57:50.953972 10470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:50.954594 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:57:50.959421 10470 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:57:50.961961 10473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:57:50.960582 10472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:50.966385 10338 server_base.cc:1047] running on GCE node
W20250904 19:57:50.967105 10476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:50.968014 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:57:50.968279 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:57:50.968448 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015870968431 us; error 0 us; skew 500 ppm
I20250904 19:57:50.969036 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:57:50.972290 10338 webserver.cc:480] Webserver started at http://127.10.24.188:38789/ using document root <none> and password file <none>
I20250904 19:57:50.972883 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:57:50.973112 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:57:50.973441 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:57:50.974232 10470 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:57:50.974776 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/master-2-root/instance:
uuid: "c19903b003444a96a9dbae99f4422ac3"
format_stamp: "Formatted at 2025-09-04 19:57:50 on dist-test-slave-f3xj"
I20250904 19:57:50.980473 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20250904 19:57:50.984702 10482 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:50.985535 10338 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20250904 19:57:50.985504 10470 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:57:50.985920 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/master-2-root
uuid: "c19903b003444a96a9dbae99f4422ac3"
format_stamp: "Formatted at 2025-09-04 19:57:50 on dist-test-slave-f3xj"
I20250904 19:57:50.986258 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20250904 19:57:50.990692 10470 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:37175: Network error: Client connection negotiation failed: client connection to 127.10.24.188:37175: connect: Connection refused (error 111)
I20250904 19:57:50.997030 10405 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } attempt: 2
I20250904 19:57:51.005285 10405 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:57:51.008802 10405 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:37175: Network error: Client connection negotiation failed: client connection to 127.10.24.188:37175: connect: Connection refused (error 111)
I20250904 19:57:51.013227 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:57:51.014417 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:57:51.033176 10470 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } attempt: 1
W20250904 19:57:51.037480 10470 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:37175: Network error: Client connection negotiation failed: client connection to 127.10.24.188:37175: connect: Connection refused (error 111)
I20250904 19:57:51.045269 10405 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } attempt: 1
W20250904 19:57:51.048978 10405 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:37175: Network error: Client connection negotiation failed: client connection to 127.10.24.188:37175: connect: Connection refused (error 111)
I20250904 19:57:51.081061 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.188:37175
I20250904 19:57:51.081167 10534 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.188:37175 every 8 connection(s)
I20250904 19:57:51.085151 10535 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:51.085125 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 19:57:51.090474 10535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:57:51.099437 10405 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } attempt: 2
I20250904 19:57:51.100950 10535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:57:51.104940 10470 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } attempt: 2
I20250904 19:57:51.112548 10535 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:57:51.120112 10405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241: Bootstrap starting.
I20250904 19:57:51.122437 10470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c: Bootstrap starting.
I20250904 19:57:51.126600 10405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:51.128414 10405 log.cc:826] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241: Log is configured to *not* fsync() on all Append() calls
I20250904 19:57:51.129310 10535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3: Bootstrap starting.
I20250904 19:57:51.129316 10470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:51.133936 10405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241: No bootstrap required, opened a new log
I20250904 19:57:51.135802 10535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:51.136626 10470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c: No bootstrap required, opened a new log
I20250904 19:57:51.141119 10535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3: No bootstrap required, opened a new log
I20250904 19:57:51.152235 10405 raft_consensus.cc:357] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0e75bb907941f8946c895ced46a241" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } } peers { permanent_uuid: "650c62ea71eb46aaa624a6aadceb250c" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } } peers { permanent_uuid: "c19903b003444a96a9dbae99f4422ac3" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } }
I20250904 19:57:51.152251 10535 raft_consensus.cc:357] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0e75bb907941f8946c895ced46a241" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } } peers { permanent_uuid: "650c62ea71eb46aaa624a6aadceb250c" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } } peers { permanent_uuid: "c19903b003444a96a9dbae99f4422ac3" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } }
I20250904 19:57:51.152251 10470 raft_consensus.cc:357] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0e75bb907941f8946c895ced46a241" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } } peers { permanent_uuid: "650c62ea71eb46aaa624a6aadceb250c" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } } peers { permanent_uuid: "c19903b003444a96a9dbae99f4422ac3" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } }
I20250904 19:57:51.152997 10405 raft_consensus.cc:383] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:51.153100 10535 raft_consensus.cc:383] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:51.153133 10470 raft_consensus.cc:383] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:51.153306 10405 raft_consensus.cc:738] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da0e75bb907941f8946c895ced46a241, State: Initialized, Role: FOLLOWER
I20250904 19:57:51.153481 10535 raft_consensus.cc:738] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c19903b003444a96a9dbae99f4422ac3, State: Initialized, Role: FOLLOWER
I20250904 19:57:51.153574 10470 raft_consensus.cc:738] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 650c62ea71eb46aaa624a6aadceb250c, State: Initialized, Role: FOLLOWER
I20250904 19:57:51.154304 10405 consensus_queue.cc:260] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0e75bb907941f8946c895ced46a241" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } } peers { permanent_uuid: "650c62ea71eb46aaa624a6aadceb250c" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } } peers { permanent_uuid: "c19903b003444a96a9dbae99f4422ac3" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } }
I20250904 19:57:51.154315 10535 consensus_queue.cc:260] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0e75bb907941f8946c895ced46a241" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } } peers { permanent_uuid: "650c62ea71eb46aaa624a6aadceb250c" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } } peers { permanent_uuid: "c19903b003444a96a9dbae99f4422ac3" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } }
I20250904 19:57:51.154313 10470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0e75bb907941f8946c895ced46a241" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } } peers { permanent_uuid: "650c62ea71eb46aaa624a6aadceb250c" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } } peers { permanent_uuid: "c19903b003444a96a9dbae99f4422ac3" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } }
I20250904 19:57:51.159977 10545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0e75bb907941f8946c895ced46a241" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } } peers { permanent_uuid: "650c62ea71eb46aaa624a6aadceb250c" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } } peers { permanent_uuid: "c19903b003444a96a9dbae99f4422ac3" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } } }
I20250904 19:57:51.160794 10545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:51.161556 10470 sys_catalog.cc:564] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:57:51.161970 10543 sys_catalog.cc:455] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0e75bb907941f8946c895ced46a241" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } } peers { permanent_uuid: "650c62ea71eb46aaa624a6aadceb250c" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } } peers { permanent_uuid: "c19903b003444a96a9dbae99f4422ac3" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } } }
I20250904 19:57:51.162734 10543 sys_catalog.cc:458] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:51.163169 10543 raft_consensus.cc:491] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:57:51.163491 10405 sys_catalog.cc:564] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:57:51.163659 10543 raft_consensus.cc:513] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0e75bb907941f8946c895ced46a241" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } } peers { permanent_uuid: "650c62ea71eb46aaa624a6aadceb250c" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } } peers { permanent_uuid: "c19903b003444a96a9dbae99f4422ac3" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } }
I20250904 19:57:51.166286 10543 leader_election.cc:290] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 650c62ea71eb46aaa624a6aadceb250c (127.10.24.189:46147), c19903b003444a96a9dbae99f4422ac3 (127.10.24.188:37175)
I20250904 19:57:51.173243 10544 sys_catalog.cc:455] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0e75bb907941f8946c895ced46a241" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } } peers { permanent_uuid: "650c62ea71eb46aaa624a6aadceb250c" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } } peers { permanent_uuid: "c19903b003444a96a9dbae99f4422ac3" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } } }
I20250904 19:57:51.174484 10535 sys_catalog.cc:564] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:57:51.174736 10510 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da0e75bb907941f8946c895ced46a241" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c19903b003444a96a9dbae99f4422ac3" is_pre_election: true
I20250904 19:57:51.175606 10510 raft_consensus.cc:2466] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da0e75bb907941f8946c895ced46a241 in term 0.
I20250904 19:57:51.177186 10356 leader_election.cc:304] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c19903b003444a96a9dbae99f4422ac3, da0e75bb907941f8946c895ced46a241; no voters:
I20250904 19:57:51.178194 10543 raft_consensus.cc:2802] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:57:51.178617 10543 raft_consensus.cc:491] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:57:51.178987 10543 raft_consensus.cc:3058] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:51.179840 10445 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da0e75bb907941f8946c895ced46a241" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "650c62ea71eb46aaa624a6aadceb250c" is_pre_election: true
I20250904 19:57:51.180487 10445 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da0e75bb907941f8946c895ced46a241 in term 0.
I20250904 19:57:51.185791 10543 raft_consensus.cc:513] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0e75bb907941f8946c895ced46a241" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } } peers { permanent_uuid: "650c62ea71eb46aaa624a6aadceb250c" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } } peers { permanent_uuid: "c19903b003444a96a9dbae99f4422ac3" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } }
I20250904 19:57:51.186460 10544 sys_catalog.cc:458] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:51.188597 10445 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da0e75bb907941f8946c895ced46a241" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "650c62ea71eb46aaa624a6aadceb250c"
I20250904 19:57:51.189090 10445 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:51.189206 10510 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "da0e75bb907941f8946c895ced46a241" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c19903b003444a96a9dbae99f4422ac3"
I20250904 19:57:51.189680 10510 raft_consensus.cc:3058] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:51.195494 10445 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da0e75bb907941f8946c895ced46a241 in term 1.
I20250904 19:57:51.195900 10510 raft_consensus.cc:2466] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da0e75bb907941f8946c895ced46a241 in term 1.
I20250904 19:57:51.196550 10357 leader_election.cc:304] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 650c62ea71eb46aaa624a6aadceb250c, da0e75bb907941f8946c895ced46a241; no voters:
I20250904 19:57:51.197152 10543 leader_election.cc:290] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [CANDIDATE]: Term 1 election: Requested vote from peers 650c62ea71eb46aaa624a6aadceb250c (127.10.24.189:46147), c19903b003444a96a9dbae99f4422ac3 (127.10.24.188:37175)
I20250904 19:57:51.197532 10546 raft_consensus.cc:2802] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:57:51.197993 10546 raft_consensus.cc:695] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [term 1 LEADER]: Becoming Leader. State: Replica: da0e75bb907941f8946c895ced46a241, State: Running, Role: LEADER
I20250904 19:57:51.198784 10546 consensus_queue.cc:237] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0e75bb907941f8946c895ced46a241" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } } peers { permanent_uuid: "650c62ea71eb46aaa624a6aadceb250c" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } } peers { permanent_uuid: "c19903b003444a96a9dbae99f4422ac3" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } }
I20250904 19:57:51.217084 10543 sys_catalog.cc:455] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da0e75bb907941f8946c895ced46a241. Latest consensus state: current_term: 1 leader_uuid: "da0e75bb907941f8946c895ced46a241" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0e75bb907941f8946c895ced46a241" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } } peers { permanent_uuid: "650c62ea71eb46aaa624a6aadceb250c" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } } peers { permanent_uuid: "c19903b003444a96a9dbae99f4422ac3" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } } }
I20250904 19:57:51.217830 10543 sys_catalog.cc:458] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [sys.catalog]: This master's current role is: LEADER
I20250904 19:57:51.226869 10567 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 19:57:51.239286 10567 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 19:57:51.256290 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20250904 19:57:51.256568 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
W20250904 19:57:51.260402 10577 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:57:51.260749 10577 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250904 19:57:51.266563 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:57:51.268309 10489 tablet.cc:2378] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20250904 19:57:51.269317 10583 catalog_manager.cc:1560] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:57:51.269598 10583 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250904 19:57:51.272708 10510 raft_consensus.cc:1273] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3 [term 1 FOLLOWER]: Refusing update from remote peer da0e75bb907941f8946c895ced46a241: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250904 19:57:51.275735 10584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:51.276160 10543 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c19903b003444a96a9dbae99f4422ac3" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:57:51.275281 10445 raft_consensus.cc:1273] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c [term 1 FOLLOWER]: Refusing update from remote peer da0e75bb907941f8946c895ced46a241: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:57:51.279239 10543 consensus_queue.cc:1035] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [LEADER]: Connected to new peer: Peer: permanent_uuid: "650c62ea71eb46aaa624a6aadceb250c" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250904 19:57:51.280310 10585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:51.304687 10545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c [sys.catalog]: SysCatalogTable state changed. Reason: New leader da0e75bb907941f8946c895ced46a241. Latest consensus state: current_term: 1 leader_uuid: "da0e75bb907941f8946c895ced46a241" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0e75bb907941f8946c895ced46a241" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } } peers { permanent_uuid: "650c62ea71eb46aaa624a6aadceb250c" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } } peers { permanent_uuid: "c19903b003444a96a9dbae99f4422ac3" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } } }
I20250904 19:57:51.305464 10545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:51.307209 10544 sys_catalog.cc:455] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da0e75bb907941f8946c895ced46a241. Latest consensus state: current_term: 1 leader_uuid: "da0e75bb907941f8946c895ced46a241" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0e75bb907941f8946c895ced46a241" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } } peers { permanent_uuid: "650c62ea71eb46aaa624a6aadceb250c" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } } peers { permanent_uuid: "c19903b003444a96a9dbae99f4422ac3" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } } }
I20250904 19:57:51.307758 10544 sys_catalog.cc:458] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:51.311851 10338 server_base.cc:1047] running on GCE node
I20250904 19:57:51.314023 10543 sys_catalog.cc:455] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da0e75bb907941f8946c895ced46a241" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0e75bb907941f8946c895ced46a241" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } } peers { permanent_uuid: "650c62ea71eb46aaa624a6aadceb250c" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } } peers { permanent_uuid: "c19903b003444a96a9dbae99f4422ac3" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } } }
I20250904 19:57:51.314688 10543 sys_catalog.cc:458] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [sys.catalog]: This master's current role is: LEADER
I20250904 19:57:51.315308 10543 sys_catalog.cc:455] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "da0e75bb907941f8946c895ced46a241" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0e75bb907941f8946c895ced46a241" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } } peers { permanent_uuid: "650c62ea71eb46aaa624a6aadceb250c" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } } peers { permanent_uuid: "c19903b003444a96a9dbae99f4422ac3" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } } }
I20250904 19:57:51.316780 10567 catalog_manager.cc:1349] Generated new cluster ID: 1b8bf8ba71924bde993a576959f57c40
I20250904 19:57:51.317070 10567 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
W20250904 19:57:51.320523 10587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:51.321689 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:57:51.321936 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:57:51.322104 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015871322092 us; error 0 us; skew 500 ppm
I20250904 19:57:51.322708 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:57:51.324571 10545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da0e75bb907941f8946c895ced46a241" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0e75bb907941f8946c895ced46a241" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } } peers { permanent_uuid: "650c62ea71eb46aaa624a6aadceb250c" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } } peers { permanent_uuid: "c19903b003444a96a9dbae99f4422ac3" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } } }
I20250904 19:57:51.325237 10545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:51.315918 10543 sys_catalog.cc:458] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [sys.catalog]: This master's current role is: LEADER
I20250904 19:57:51.350643 10544 sys_catalog.cc:455] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "da0e75bb907941f8946c895ced46a241" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da0e75bb907941f8946c895ced46a241" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 44751 } } peers { permanent_uuid: "650c62ea71eb46aaa624a6aadceb250c" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46147 } } peers { permanent_uuid: "c19903b003444a96a9dbae99f4422ac3" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37175 } } }
I20250904 19:57:51.351379 10544 sys_catalog.cc:458] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:51.351882 10338 webserver.cc:480] Webserver started at http://127.10.24.129:39797/ using document root <none> and password file <none>
I20250904 19:57:51.352492 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:57:51.352694 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:57:51.352977 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:57:51.354329 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/ts-0-root/instance:
uuid: "dd0d980a231c4ccca173b477f06233d4"
format_stamp: "Formatted at 2025-09-04 19:57:51 on dist-test-slave-f3xj"
I20250904 19:57:51.368757 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.014s user 0.001s sys 0.012s
I20250904 19:57:51.372251 10567 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 19:57:51.373140 10600 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:51.374063 10338 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20250904 19:57:51.374493 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/ts-0-root
uuid: "dd0d980a231c4ccca173b477f06233d4"
format_stamp: "Formatted at 2025-09-04 19:57:51 on dist-test-slave-f3xj"
I20250904 19:57:51.374908 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:57:51.374114 10567 catalog_manager.cc:1506] Loading token signing keys...
I20250904 19:57:51.410666 10567 catalog_manager.cc:5955] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241: Generated new TSK 0
I20250904 19:57:51.411751 10567 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 19:57:51.412709 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:57:51.414423 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:57:51.416970 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:57:51.420794 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:57:51.421001 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:51.421252 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:57:51.421415 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:51.516503 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.129:37523
I20250904 19:57:51.516731 10662 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.129:37523 every 8 connection(s)
I20250904 19:57:51.548214 10663 heartbeater.cc:344] Connected to a master server at 127.10.24.189:46147
I20250904 19:57:51.548660 10663 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:51.549770 10663 heartbeater.cc:507] Master 127.10.24.189:46147 requested a full tablet report, sending...
I20250904 19:57:51.551486 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:57:51.553890 10435 ts_manager.cc:194] Registered new tserver with Master: dd0d980a231c4ccca173b477f06233d4 (127.10.24.129:37523)
I20250904 19:57:51.556802 10664 heartbeater.cc:344] Connected to a master server at 127.10.24.188:37175
I20250904 19:57:51.557206 10664 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:51.558203 10664 heartbeater.cc:507] Master 127.10.24.188:37175 requested a full tablet report, sending...
I20250904 19:57:51.561367 10500 ts_manager.cc:194] Registered new tserver with Master: dd0d980a231c4ccca173b477f06233d4 (127.10.24.129:37523)
W20250904 19:57:51.565793 10670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:51.572060 10665 heartbeater.cc:344] Connected to a master server at 127.10.24.190:44751
I20250904 19:57:51.572468 10665 heartbeater.cc:461] Registering TS with master...
W20250904 19:57:51.573395 10671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:51.577682 10665 heartbeater.cc:507] Master 127.10.24.190:44751 requested a full tablet report, sending...
I20250904 19:57:51.580150 10370 ts_manager.cc:194] Registered new tserver with Master: dd0d980a231c4ccca173b477f06233d4 (127.10.24.129:37523)
I20250904 19:57:51.581035 10338 server_base.cc:1047] running on GCE node
W20250904 19:57:51.582033 10673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:51.583099 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:57:51.583348 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:57:51.583284 10370 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57782
I20250904 19:57:51.583650 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015871583632 us; error 0 us; skew 500 ppm
I20250904 19:57:51.584430 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:57:51.588318 10338 webserver.cc:480] Webserver started at http://127.10.24.130:34401/ using document root <none> and password file <none>
I20250904 19:57:51.588915 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:57:51.589152 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:57:51.589447 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:57:51.590745 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/ts-1-root/instance:
uuid: "7cd41621ab324b509ab8481e8a896b5e"
format_stamp: "Formatted at 2025-09-04 19:57:51 on dist-test-slave-f3xj"
I20250904 19:57:51.596417 10665 heartbeater.cc:499] Master 127.10.24.190:44751 was elected leader, sending a full tablet report...
I20250904 19:57:51.596483 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20250904 19:57:51.601145 10678 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:51.601872 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250904 19:57:51.602238 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/ts-1-root
uuid: "7cd41621ab324b509ab8481e8a896b5e"
format_stamp: "Formatted at 2025-09-04 19:57:51 on dist-test-slave-f3xj"
I20250904 19:57:51.602624 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:57:51.632582 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:57:51.633697 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:57:51.635360 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:57:51.637856 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:57:51.638051 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:51.638275 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:57:51.638474 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:51.734869 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.130:37445
I20250904 19:57:51.734987 10740 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.130:37445 every 8 connection(s)
I20250904 19:57:51.756582 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:57:51.771539 10741 heartbeater.cc:344] Connected to a master server at 127.10.24.189:46147
I20250904 19:57:51.771973 10741 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:51.772100 10742 heartbeater.cc:344] Connected to a master server at 127.10.24.188:37175
I20250904 19:57:51.772511 10742 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:51.772980 10741 heartbeater.cc:507] Master 127.10.24.189:46147 requested a full tablet report, sending...
I20250904 19:57:51.773361 10742 heartbeater.cc:507] Master 127.10.24.188:37175 requested a full tablet report, sending...
I20250904 19:57:51.775926 10500 ts_manager.cc:194] Registered new tserver with Master: 7cd41621ab324b509ab8481e8a896b5e (127.10.24.130:37445)
I20250904 19:57:51.775866 10435 ts_manager.cc:194] Registered new tserver with Master: 7cd41621ab324b509ab8481e8a896b5e (127.10.24.130:37445)
W20250904 19:57:51.786976 10748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:51.789047 10743 heartbeater.cc:344] Connected to a master server at 127.10.24.190:44751
I20250904 19:57:51.789486 10743 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:51.790367 10743 heartbeater.cc:507] Master 127.10.24.190:44751 requested a full tablet report, sending...
W20250904 19:57:51.794723 10749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:51.798383 10370 ts_manager.cc:194] Registered new tserver with Master: 7cd41621ab324b509ab8481e8a896b5e (127.10.24.130:37445)
I20250904 19:57:51.800282 10370 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57792
I20250904 19:57:51.805845 10338 server_base.cc:1047] running on GCE node
W20250904 19:57:51.812536 10752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:51.813723 10743 heartbeater.cc:499] Master 127.10.24.190:44751 was elected leader, sending a full tablet report...
I20250904 19:57:51.816323 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:57:51.816555 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:57:51.816718 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015871816701 us; error 0 us; skew 500 ppm
I20250904 19:57:51.817255 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:57:51.826457 10338 webserver.cc:480] Webserver started at http://127.10.24.131:38571/ using document root <none> and password file <none>
I20250904 19:57:51.827248 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:57:51.827541 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:57:51.827903 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:57:51.829305 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/ts-2-root/instance:
uuid: "65dc647c385342abbec3c69781812c7e"
format_stamp: "Formatted at 2025-09-04 19:57:51 on dist-test-slave-f3xj"
I20250904 19:57:51.835001 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20250904 19:57:51.839010 10756 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:51.839797 10338 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20250904 19:57:51.840134 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/ts-2-root
uuid: "65dc647c385342abbec3c69781812c7e"
format_stamp: "Formatted at 2025-09-04 19:57:51 on dist-test-slave-f3xj"
I20250904 19:57:51.840548 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterOk.1757015869512910-10338-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:57:51.889161 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:57:51.890846 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:57:51.896631 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:57:51.898916 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:57:51.899209 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20250904 19:57:51.899497 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:57:51.899698 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:52.061030 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.131:42491
I20250904 19:57:52.061131 10818 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.131:42491 every 8 connection(s)
I20250904 19:57:52.125617 10822 heartbeater.cc:344] Connected to a master server at 127.10.24.190:44751
I20250904 19:57:52.126053 10822 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:52.126995 10822 heartbeater.cc:507] Master 127.10.24.190:44751 requested a full tablet report, sending...
I20250904 19:57:52.129155 10370 ts_manager.cc:194] Registered new tserver with Master: 65dc647c385342abbec3c69781812c7e (127.10.24.131:42491)
I20250904 19:57:52.130813 10370 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57802
I20250904 19:57:52.133678 10819 heartbeater.cc:344] Connected to a master server at 127.10.24.189:46147
I20250904 19:57:52.134059 10819 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:52.140239 10819 heartbeater.cc:507] Master 127.10.24.189:46147 requested a full tablet report, sending...
I20250904 19:57:52.143587 10435 ts_manager.cc:194] Registered new tserver with Master: 65dc647c385342abbec3c69781812c7e (127.10.24.131:42491)
I20250904 19:57:52.144451 10820 heartbeater.cc:344] Connected to a master server at 127.10.24.188:37175
I20250904 19:57:52.144937 10820 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:52.145628 10820 heartbeater.cc:507] Master 127.10.24.188:37175 requested a full tablet report, sending...
I20250904 19:57:52.148001 10499 ts_manager.cc:194] Registered new tserver with Master: 65dc647c385342abbec3c69781812c7e (127.10.24.131:42491)
I20250904 19:57:52.147964 10338 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.028159991s
I20250904 19:57:52.151549 10822 heartbeater.cc:499] Master 127.10.24.190:44751 was elected leader, sending a full tablet report...
I20250904 19:57:52.224270 10366 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57812:
name: "ksck-test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\n\000\000\000\004\001\024\000\000\000""\004\001\n\000\000\000\004\001\024\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 19:57:52.262681 10366 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table ksck-test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 19:57:52.269780 10577 catalog_manager.cc:1261] Loaded cluster ID: 1b8bf8ba71924bde993a576959f57c40
I20250904 19:57:52.270125 10577 catalog_manager.cc:1554] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c: loading cluster ID for follower catalog manager: success
I20250904 19:57:52.273829 10583 catalog_manager.cc:1261] Loaded cluster ID: 1b8bf8ba71924bde993a576959f57c40
I20250904 19:57:52.274080 10583 catalog_manager.cc:1554] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3: loading cluster ID for follower catalog manager: success
I20250904 19:57:52.286554 10583 catalog_manager.cc:1576] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3: acquiring CA information for follower catalog manager: success
I20250904 19:57:52.298933 10577 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c: acquiring CA information for follower catalog manager: success
I20250904 19:57:52.310536 10577 catalog_manager.cc:1604] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:57:52.310686 10583 catalog_manager.cc:1604] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:57:52.490031 10706 tablet_service.cc:1468] Processing CreateTablet for tablet ace90b31faeb4a63a2791ce38c5821c1 (DEFAULT_TABLE table=ksck-test-table [id=3f0ddcd529554420b6319ec1ec13e1d0]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:57:52.491765 10706 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ace90b31faeb4a63a2791ce38c5821c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:52.493129 10704 tablet_service.cc:1468] Processing CreateTablet for tablet 3d7958cd6f844eab8f4406cffa79be43 (DEFAULT_TABLE table=ksck-test-table [id=3f0ddcd529554420b6319ec1ec13e1d0]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:57:52.491350 10705 tablet_service.cc:1468] Processing CreateTablet for tablet cf72ca7d4cb4466fb0ed0305e7f82dba (DEFAULT_TABLE table=ksck-test-table [id=3f0ddcd529554420b6319ec1ec13e1d0]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:57:52.494486 10704 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d7958cd6f844eab8f4406cffa79be43. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:52.499691 10705 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf72ca7d4cb4466fb0ed0305e7f82dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:52.529255 10784 tablet_service.cc:1468] Processing CreateTablet for tablet ace90b31faeb4a63a2791ce38c5821c1 (DEFAULT_TABLE table=ksck-test-table [id=3f0ddcd529554420b6319ec1ec13e1d0]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:57:52.532289 10784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ace90b31faeb4a63a2791ce38c5821c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:52.533568 10782 tablet_service.cc:1468] Processing CreateTablet for tablet 3d7958cd6f844eab8f4406cffa79be43 (DEFAULT_TABLE table=ksck-test-table [id=3f0ddcd529554420b6319ec1ec13e1d0]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:57:52.535408 10782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d7958cd6f844eab8f4406cffa79be43. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:52.533532 10783 tablet_service.cc:1468] Processing CreateTablet for tablet cf72ca7d4cb4466fb0ed0305e7f82dba (DEFAULT_TABLE table=ksck-test-table [id=3f0ddcd529554420b6319ec1ec13e1d0]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:57:52.539525 10783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf72ca7d4cb4466fb0ed0305e7f82dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:52.557857 10628 tablet_service.cc:1468] Processing CreateTablet for tablet ace90b31faeb4a63a2791ce38c5821c1 (DEFAULT_TABLE table=ksck-test-table [id=3f0ddcd529554420b6319ec1ec13e1d0]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:57:52.560110 10628 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ace90b31faeb4a63a2791ce38c5821c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:52.579375 10627 tablet_service.cc:1468] Processing CreateTablet for tablet cf72ca7d4cb4466fb0ed0305e7f82dba (DEFAULT_TABLE table=ksck-test-table [id=3f0ddcd529554420b6319ec1ec13e1d0]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:57:52.582028 10627 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf72ca7d4cb4466fb0ed0305e7f82dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:52.596455 10626 tablet_service.cc:1468] Processing CreateTablet for tablet 3d7958cd6f844eab8f4406cffa79be43 (DEFAULT_TABLE table=ksck-test-table [id=3f0ddcd529554420b6319ec1ec13e1d0]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:57:52.597775 10626 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d7958cd6f844eab8f4406cffa79be43. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:52.671435 10848 tablet_bootstrap.cc:492] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e: Bootstrap starting.
I20250904 19:57:52.697134 10849 tablet_bootstrap.cc:492] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e: Bootstrap starting.
I20250904 19:57:52.697950 10848 tablet_bootstrap.cc:654] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:52.705907 10850 tablet_bootstrap.cc:492] T cf72ca7d4cb4466fb0ed0305e7f82dba P dd0d980a231c4ccca173b477f06233d4: Bootstrap starting.
I20250904 19:57:52.714401 10849 tablet_bootstrap.cc:654] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:52.736488 10850 tablet_bootstrap.cc:654] T cf72ca7d4cb4466fb0ed0305e7f82dba P dd0d980a231c4ccca173b477f06233d4: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:52.749181 10849 tablet_bootstrap.cc:492] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e: No bootstrap required, opened a new log
I20250904 19:57:52.749810 10849 ts_tablet_manager.cc:1397] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e: Time spent bootstrapping tablet: real 0.058s user 0.011s sys 0.002s
I20250904 19:57:52.752080 10848 tablet_bootstrap.cc:492] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e: No bootstrap required, opened a new log
I20250904 19:57:52.752673 10848 ts_tablet_manager.cc:1397] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e: Time spent bootstrapping tablet: real 0.085s user 0.017s sys 0.007s
I20250904 19:57:52.753005 10850 tablet_bootstrap.cc:492] T cf72ca7d4cb4466fb0ed0305e7f82dba P dd0d980a231c4ccca173b477f06233d4: No bootstrap required, opened a new log
I20250904 19:57:52.753448 10849 raft_consensus.cc:357] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:52.754273 10849 raft_consensus.cc:383] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:52.754549 10849 raft_consensus.cc:738] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd41621ab324b509ab8481e8a896b5e, State: Initialized, Role: FOLLOWER
I20250904 19:57:52.755236 10849 consensus_queue.cc:260] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:52.763725 10850 ts_tablet_manager.cc:1397] T cf72ca7d4cb4466fb0ed0305e7f82dba P dd0d980a231c4ccca173b477f06233d4: Time spent bootstrapping tablet: real 0.067s user 0.008s sys 0.004s
I20250904 19:57:52.766590 10850 raft_consensus.cc:357] T cf72ca7d4cb4466fb0ed0305e7f82dba P dd0d980a231c4ccca173b477f06233d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:52.767431 10850 raft_consensus.cc:383] T cf72ca7d4cb4466fb0ed0305e7f82dba P dd0d980a231c4ccca173b477f06233d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:52.767884 10850 raft_consensus.cc:738] T cf72ca7d4cb4466fb0ed0305e7f82dba P dd0d980a231c4ccca173b477f06233d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd0d980a231c4ccca173b477f06233d4, State: Initialized, Role: FOLLOWER
I20250904 19:57:52.768690 10850 consensus_queue.cc:260] T cf72ca7d4cb4466fb0ed0305e7f82dba P dd0d980a231c4ccca173b477f06233d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:52.765483 10848 raft_consensus.cc:357] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:52.773430 10848 raft_consensus.cc:383] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:52.773995 10848 raft_consensus.cc:738] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dc647c385342abbec3c69781812c7e, State: Initialized, Role: FOLLOWER
I20250904 19:57:52.780599 10848 consensus_queue.cc:260] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:52.797242 10848 ts_tablet_manager.cc:1428] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e: Time spent starting tablet: real 0.044s user 0.006s sys 0.002s
I20250904 19:57:52.790284 10850 ts_tablet_manager.cc:1428] T cf72ca7d4cb4466fb0ed0305e7f82dba P dd0d980a231c4ccca173b477f06233d4: Time spent starting tablet: real 0.026s user 0.006s sys 0.004s
I20250904 19:57:52.818797 10850 tablet_bootstrap.cc:492] T ace90b31faeb4a63a2791ce38c5821c1 P dd0d980a231c4ccca173b477f06233d4: Bootstrap starting.
I20250904 19:57:52.830973 10848 tablet_bootstrap.cc:492] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e: Bootstrap starting.
I20250904 19:57:52.832299 10856 raft_consensus.cc:491] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:57:52.832792 10856 raft_consensus.cc:513] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:52.864619 10850 tablet_bootstrap.cc:654] T ace90b31faeb4a63a2791ce38c5821c1 P dd0d980a231c4ccca173b477f06233d4: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:52.871798 10856 leader_election.cc:290] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd0d980a231c4ccca173b477f06233d4 (127.10.24.129:37523), 7cd41621ab324b509ab8481e8a896b5e (127.10.24.130:37445)
I20250904 19:57:52.888088 10849 ts_tablet_manager.cc:1428] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e: Time spent starting tablet: real 0.138s user 0.012s sys 0.018s
I20250904 19:57:52.889165 10849 tablet_bootstrap.cc:492] T ace90b31faeb4a63a2791ce38c5821c1 P 7cd41621ab324b509ab8481e8a896b5e: Bootstrap starting.
I20250904 19:57:52.879099 10857 raft_consensus.cc:491] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:57:52.893913 10857 raft_consensus.cc:513] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:52.885309 10848 tablet_bootstrap.cc:654] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:52.906862 10857 leader_election.cc:290] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd0d980a231c4ccca173b477f06233d4 (127.10.24.129:37523), 65dc647c385342abbec3c69781812c7e (127.10.24.131:42491)
I20250904 19:57:52.945277 10849 tablet_bootstrap.cc:654] T ace90b31faeb4a63a2791ce38c5821c1 P 7cd41621ab324b509ab8481e8a896b5e: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:52.957286 10848 tablet_bootstrap.cc:492] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e: No bootstrap required, opened a new log
I20250904 19:57:52.957881 10848 ts_tablet_manager.cc:1397] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e: Time spent bootstrapping tablet: real 0.127s user 0.016s sys 0.018s
I20250904 19:57:52.961299 10638 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3d7958cd6f844eab8f4406cffa79be43" candidate_uuid: "65dc647c385342abbec3c69781812c7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd0d980a231c4ccca173b477f06233d4" is_pre_election: true
W20250904 19:57:52.970789 10757 leader_election.cc:343] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer dd0d980a231c4ccca173b477f06233d4 (127.10.24.129:37523): Illegal state: must be running to vote when last-logged opid is not known
I20250904 19:57:52.977614 10848 raft_consensus.cc:357] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:52.979820 10716 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3d7958cd6f844eab8f4406cffa79be43" candidate_uuid: "65dc647c385342abbec3c69781812c7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd41621ab324b509ab8481e8a896b5e" is_pre_election: true
I20250904 19:57:52.980456 10848 raft_consensus.cc:383] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20250904 19:57:52.984634 10758 leader_election.cc:343] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cd41621ab324b509ab8481e8a896b5e (127.10.24.130:37445): Illegal state: must be running to vote when last-logged opid is not known
I20250904 19:57:52.987232 10758 leader_election.cc:304] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 65dc647c385342abbec3c69781812c7e; no voters: 7cd41621ab324b509ab8481e8a896b5e, dd0d980a231c4ccca173b477f06233d4
I20250904 19:57:52.989001 10856 raft_consensus.cc:2747] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250904 19:57:52.990351 10849 tablet_bootstrap.cc:492] T ace90b31faeb4a63a2791ce38c5821c1 P 7cd41621ab324b509ab8481e8a896b5e: No bootstrap required, opened a new log
I20250904 19:57:52.987672 10848 raft_consensus.cc:738] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dc647c385342abbec3c69781812c7e, State: Initialized, Role: FOLLOWER
I20250904 19:57:52.993036 10848 consensus_queue.cc:260] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:52.990952 10849 ts_tablet_manager.cc:1397] T ace90b31faeb4a63a2791ce38c5821c1 P 7cd41621ab324b509ab8481e8a896b5e: Time spent bootstrapping tablet: real 0.102s user 0.009s sys 0.022s
I20250904 19:57:52.994693 10850 tablet_bootstrap.cc:492] T ace90b31faeb4a63a2791ce38c5821c1 P dd0d980a231c4ccca173b477f06233d4: No bootstrap required, opened a new log
I20250904 19:57:52.995208 10638 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "cf72ca7d4cb4466fb0ed0305e7f82dba" candidate_uuid: "7cd41621ab324b509ab8481e8a896b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd0d980a231c4ccca173b477f06233d4" is_pre_election: true
I20250904 19:57:52.996155 10638 raft_consensus.cc:2466] T cf72ca7d4cb4466fb0ed0305e7f82dba P dd0d980a231c4ccca173b477f06233d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cd41621ab324b509ab8481e8a896b5e in term 0.
I20250904 19:57:52.996146 10850 ts_tablet_manager.cc:1397] T ace90b31faeb4a63a2791ce38c5821c1 P dd0d980a231c4ccca173b477f06233d4: Time spent bootstrapping tablet: real 0.178s user 0.013s sys 0.010s
I20250904 19:57:52.997452 10679 leader_election.cc:304] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cd41621ab324b509ab8481e8a896b5e, dd0d980a231c4ccca173b477f06233d4; no voters:
I20250904 19:57:52.998342 10857 raft_consensus.cc:2802] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:57:52.998684 10857 raft_consensus.cc:491] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:57:52.999045 10857 raft_consensus.cc:3058] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:52.998829 10849 raft_consensus.cc:357] T ace90b31faeb4a63a2791ce38c5821c1 P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:52.999200 10794 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "cf72ca7d4cb4466fb0ed0305e7f82dba" candidate_uuid: "7cd41621ab324b509ab8481e8a896b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dc647c385342abbec3c69781812c7e" is_pre_election: true
I20250904 19:57:52.999531 10849 raft_consensus.cc:383] T ace90b31faeb4a63a2791ce38c5821c1 P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:52.999920 10849 raft_consensus.cc:738] T ace90b31faeb4a63a2791ce38c5821c1 P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd41621ab324b509ab8481e8a896b5e, State: Initialized, Role: FOLLOWER
W20250904 19:57:53.000741 10679 leader_election.cc:343] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 65dc647c385342abbec3c69781812c7e (127.10.24.131:42491): Illegal state: must be running to vote when last-logged opid is not known
I20250904 19:57:53.000574 10849 consensus_queue.cc:260] T ace90b31faeb4a63a2791ce38c5821c1 P 7cd41621ab324b509ab8481e8a896b5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:53.002939 10849 ts_tablet_manager.cc:1428] T ace90b31faeb4a63a2791ce38c5821c1 P 7cd41621ab324b509ab8481e8a896b5e: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20250904 19:57:53.009241 10849 tablet_bootstrap.cc:492] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e: Bootstrap starting.
I20250904 19:57:53.010814 10857 raft_consensus.cc:513] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:53.012646 10857 leader_election.cc:290] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [CANDIDATE]: Term 1 election: Requested vote from peers dd0d980a231c4ccca173b477f06233d4 (127.10.24.129:37523), 65dc647c385342abbec3c69781812c7e (127.10.24.131:42491)
I20250904 19:57:53.016498 10794 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "cf72ca7d4cb4466fb0ed0305e7f82dba" candidate_uuid: "7cd41621ab324b509ab8481e8a896b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dc647c385342abbec3c69781812c7e"
W20250904 19:57:53.017753 10679 leader_election.cc:343] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 65dc647c385342abbec3c69781812c7e (127.10.24.131:42491): Illegal state: must be running to vote when last-logged opid is not known
I20250904 19:57:53.018489 10638 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "cf72ca7d4cb4466fb0ed0305e7f82dba" candidate_uuid: "7cd41621ab324b509ab8481e8a896b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd0d980a231c4ccca173b477f06233d4"
I20250904 19:57:53.019057 10638 raft_consensus.cc:3058] T cf72ca7d4cb4466fb0ed0305e7f82dba P dd0d980a231c4ccca173b477f06233d4 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:53.027128 10638 raft_consensus.cc:2466] T cf72ca7d4cb4466fb0ed0305e7f82dba P dd0d980a231c4ccca173b477f06233d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd41621ab324b509ab8481e8a896b5e in term 1.
I20250904 19:57:53.028745 10679 leader_election.cc:304] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7cd41621ab324b509ab8481e8a896b5e, dd0d980a231c4ccca173b477f06233d4; no voters: 65dc647c385342abbec3c69781812c7e
I20250904 19:57:53.027024 10850 raft_consensus.cc:357] T ace90b31faeb4a63a2791ce38c5821c1 P dd0d980a231c4ccca173b477f06233d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:53.029623 10850 raft_consensus.cc:383] T ace90b31faeb4a63a2791ce38c5821c1 P dd0d980a231c4ccca173b477f06233d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:53.030006 10850 raft_consensus.cc:738] T ace90b31faeb4a63a2791ce38c5821c1 P dd0d980a231c4ccca173b477f06233d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd0d980a231c4ccca173b477f06233d4, State: Initialized, Role: FOLLOWER
I20250904 19:57:53.029896 10857 raft_consensus.cc:2802] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:57:53.028748 10848 ts_tablet_manager.cc:1428] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e: Time spent starting tablet: real 0.071s user 0.009s sys 0.001s
I20250904 19:57:53.032384 10857 raft_consensus.cc:695] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [term 1 LEADER]: Becoming Leader. State: Replica: 7cd41621ab324b509ab8481e8a896b5e, State: Running, Role: LEADER
I20250904 19:57:53.033226 10857 consensus_queue.cc:237] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:53.033912 10849 tablet_bootstrap.cc:654] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:53.035811 10850 consensus_queue.cc:260] T ace90b31faeb4a63a2791ce38c5821c1 P dd0d980a231c4ccca173b477f06233d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:53.035540 10848 tablet_bootstrap.cc:492] T cf72ca7d4cb4466fb0ed0305e7f82dba P 65dc647c385342abbec3c69781812c7e: Bootstrap starting.
I20250904 19:57:53.043011 10850 ts_tablet_manager.cc:1428] T ace90b31faeb4a63a2791ce38c5821c1 P dd0d980a231c4ccca173b477f06233d4: Time spent starting tablet: real 0.046s user 0.004s sys 0.002s
I20250904 19:57:53.044065 10850 tablet_bootstrap.cc:492] T 3d7958cd6f844eab8f4406cffa79be43 P dd0d980a231c4ccca173b477f06233d4: Bootstrap starting.
I20250904 19:57:53.053048 10850 tablet_bootstrap.cc:654] T 3d7958cd6f844eab8f4406cffa79be43 P dd0d980a231c4ccca173b477f06233d4: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:53.054230 10848 tablet_bootstrap.cc:654] T cf72ca7d4cb4466fb0ed0305e7f82dba P 65dc647c385342abbec3c69781812c7e: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:53.065896 10849 tablet_bootstrap.cc:492] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e: No bootstrap required, opened a new log
I20250904 19:57:53.077282 10849 ts_tablet_manager.cc:1397] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e: Time spent bootstrapping tablet: real 0.068s user 0.011s sys 0.000s
I20250904 19:57:53.097579 10856 raft_consensus.cc:491] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:57:53.098095 10856 raft_consensus.cc:513] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:53.099949 10856 leader_election.cc:290] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cd41621ab324b509ab8481e8a896b5e (127.10.24.130:37445), dd0d980a231c4ccca173b477f06233d4 (127.10.24.129:37523)
I20250904 19:57:53.101461 10716 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "ace90b31faeb4a63a2791ce38c5821c1" candidate_uuid: "65dc647c385342abbec3c69781812c7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd41621ab324b509ab8481e8a896b5e" is_pre_election: true
I20250904 19:57:53.102499 10368 catalog_manager.cc:5582] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cd41621ab324b509ab8481e8a896b5e (127.10.24.130). New cstate: current_term: 1 leader_uuid: "7cd41621ab324b509ab8481e8a896b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } health_report { overall_health: UNKNOWN } } }
I20250904 19:57:53.103312 10716 raft_consensus.cc:2466] T ace90b31faeb4a63a2791ce38c5821c1 P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65dc647c385342abbec3c69781812c7e in term 0.
I20250904 19:57:53.096364 10849 raft_consensus.cc:357] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:53.107694 10758 leader_election.cc:304] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dc647c385342abbec3c69781812c7e, 7cd41621ab324b509ab8481e8a896b5e; no voters:
I20250904 19:57:53.110880 10849 raft_consensus.cc:383] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:53.111416 10849 raft_consensus.cc:738] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cd41621ab324b509ab8481e8a896b5e, State: Initialized, Role: FOLLOWER
I20250904 19:57:53.113561 10849 consensus_queue.cc:260] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:53.110337 10848 tablet_bootstrap.cc:492] T cf72ca7d4cb4466fb0ed0305e7f82dba P 65dc647c385342abbec3c69781812c7e: No bootstrap required, opened a new log
I20250904 19:57:53.114738 10848 ts_tablet_manager.cc:1397] T cf72ca7d4cb4466fb0ed0305e7f82dba P 65dc647c385342abbec3c69781812c7e: Time spent bootstrapping tablet: real 0.079s user 0.008s sys 0.014s
I20250904 19:57:53.117581 10848 raft_consensus.cc:357] T cf72ca7d4cb4466fb0ed0305e7f82dba P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:53.118319 10848 raft_consensus.cc:383] T cf72ca7d4cb4466fb0ed0305e7f82dba P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:53.118615 10848 raft_consensus.cc:738] T cf72ca7d4cb4466fb0ed0305e7f82dba P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dc647c385342abbec3c69781812c7e, State: Initialized, Role: FOLLOWER
I20250904 19:57:53.119300 10856 raft_consensus.cc:2802] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:57:53.119719 10856 raft_consensus.cc:491] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:57:53.120081 10856 raft_consensus.cc:3058] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:53.121112 10849 ts_tablet_manager.cc:1428] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e: Time spent starting tablet: real 0.044s user 0.007s sys 0.000s
I20250904 19:57:53.111513 10638 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "ace90b31faeb4a63a2791ce38c5821c1" candidate_uuid: "65dc647c385342abbec3c69781812c7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd0d980a231c4ccca173b477f06233d4" is_pre_election: true
I20250904 19:57:53.129316 10638 raft_consensus.cc:2466] T ace90b31faeb4a63a2791ce38c5821c1 P dd0d980a231c4ccca173b477f06233d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65dc647c385342abbec3c69781812c7e in term 0.
I20250904 19:57:53.131960 10850 tablet_bootstrap.cc:492] T 3d7958cd6f844eab8f4406cffa79be43 P dd0d980a231c4ccca173b477f06233d4: No bootstrap required, opened a new log
I20250904 19:57:53.132524 10850 ts_tablet_manager.cc:1397] T 3d7958cd6f844eab8f4406cffa79be43 P dd0d980a231c4ccca173b477f06233d4: Time spent bootstrapping tablet: real 0.089s user 0.023s sys 0.012s
I20250904 19:57:53.135447 10850 raft_consensus.cc:357] T 3d7958cd6f844eab8f4406cffa79be43 P dd0d980a231c4ccca173b477f06233d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:53.135891 10848 consensus_queue.cc:260] T cf72ca7d4cb4466fb0ed0305e7f82dba P 65dc647c385342abbec3c69781812c7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:53.136494 10850 raft_consensus.cc:383] T 3d7958cd6f844eab8f4406cffa79be43 P dd0d980a231c4ccca173b477f06233d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:53.136951 10850 raft_consensus.cc:738] T 3d7958cd6f844eab8f4406cffa79be43 P dd0d980a231c4ccca173b477f06233d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd0d980a231c4ccca173b477f06233d4, State: Initialized, Role: FOLLOWER
I20250904 19:57:53.137663 10850 consensus_queue.cc:260] T 3d7958cd6f844eab8f4406cffa79be43 P dd0d980a231c4ccca173b477f06233d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:53.140136 10850 ts_tablet_manager.cc:1428] T 3d7958cd6f844eab8f4406cffa79be43 P dd0d980a231c4ccca173b477f06233d4: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250904 19:57:53.139916 10856 raft_consensus.cc:513] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:53.157850 10856 leader_election.cc:290] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [CANDIDATE]: Term 1 election: Requested vote from peers 7cd41621ab324b509ab8481e8a896b5e (127.10.24.130:37445), dd0d980a231c4ccca173b477f06233d4 (127.10.24.129:37523)
I20250904 19:57:53.158550 10716 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "ace90b31faeb4a63a2791ce38c5821c1" candidate_uuid: "65dc647c385342abbec3c69781812c7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd41621ab324b509ab8481e8a896b5e"
I20250904 19:57:53.159145 10716 raft_consensus.cc:3058] T ace90b31faeb4a63a2791ce38c5821c1 P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:53.160313 10638 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "ace90b31faeb4a63a2791ce38c5821c1" candidate_uuid: "65dc647c385342abbec3c69781812c7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd0d980a231c4ccca173b477f06233d4"
I20250904 19:57:53.160878 10638 raft_consensus.cc:3058] T ace90b31faeb4a63a2791ce38c5821c1 P dd0d980a231c4ccca173b477f06233d4 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:53.166661 10716 raft_consensus.cc:2466] T ace90b31faeb4a63a2791ce38c5821c1 P 7cd41621ab324b509ab8481e8a896b5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dc647c385342abbec3c69781812c7e in term 1.
I20250904 19:57:53.168004 10758 leader_election.cc:304] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dc647c385342abbec3c69781812c7e, 7cd41621ab324b509ab8481e8a896b5e; no voters:
I20250904 19:57:53.172672 10638 raft_consensus.cc:2466] T ace90b31faeb4a63a2791ce38c5821c1 P dd0d980a231c4ccca173b477f06233d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65dc647c385342abbec3c69781812c7e in term 1.
I20250904 19:57:53.173000 10857 raft_consensus.cc:491] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:57:53.173786 10857 raft_consensus.cc:513] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:53.177012 10637 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3d7958cd6f844eab8f4406cffa79be43" candidate_uuid: "7cd41621ab324b509ab8481e8a896b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd0d980a231c4ccca173b477f06233d4" is_pre_election: true
I20250904 19:57:53.177726 10637 raft_consensus.cc:2466] T 3d7958cd6f844eab8f4406cffa79be43 P dd0d980a231c4ccca173b477f06233d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cd41621ab324b509ab8481e8a896b5e in term 0.
I20250904 19:57:53.179059 10679 leader_election.cc:304] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7cd41621ab324b509ab8481e8a896b5e, dd0d980a231c4ccca173b477f06233d4; no voters:
I20250904 19:57:53.180016 10854 raft_consensus.cc:2802] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:57:53.180394 10854 raft_consensus.cc:491] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:57:53.180713 10854 raft_consensus.cc:3058] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:53.183900 10857 leader_election.cc:290] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd0d980a231c4ccca173b477f06233d4 (127.10.24.129:37523), 65dc647c385342abbec3c69781812c7e (127.10.24.131:42491)
I20250904 19:57:53.197980 10794 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3d7958cd6f844eab8f4406cffa79be43" candidate_uuid: "7cd41621ab324b509ab8481e8a896b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dc647c385342abbec3c69781812c7e" is_pre_election: true
I20250904 19:57:53.198710 10794 raft_consensus.cc:2466] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cd41621ab324b509ab8481e8a896b5e in term 0.
I20250904 19:57:53.200953 10854 raft_consensus.cc:513] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:53.180747 10875 raft_consensus.cc:491] T cf72ca7d4cb4466fb0ed0305e7f82dba P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:57:53.209779 10875 raft_consensus.cc:513] T cf72ca7d4cb4466fb0ed0305e7f82dba P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:53.211467 10794 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3d7958cd6f844eab8f4406cffa79be43" candidate_uuid: "7cd41621ab324b509ab8481e8a896b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dc647c385342abbec3c69781812c7e"
I20250904 19:57:53.212067 10794 raft_consensus.cc:3058] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:53.218048 10856 raft_consensus.cc:2802] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:57:53.218531 10856 raft_consensus.cc:695] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [term 1 LEADER]: Becoming Leader. State: Replica: 65dc647c385342abbec3c69781812c7e, State: Running, Role: LEADER
I20250904 19:57:53.219321 10856 consensus_queue.cc:237] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:53.224434 10794 raft_consensus.cc:2466] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd41621ab324b509ab8481e8a896b5e in term 1.
I20250904 19:57:53.226789 10848 ts_tablet_manager.cc:1428] T cf72ca7d4cb4466fb0ed0305e7f82dba P 65dc647c385342abbec3c69781812c7e: Time spent starting tablet: real 0.112s user 0.006s sys 0.006s
I20250904 19:57:53.227855 10679 leader_election.cc:304] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65dc647c385342abbec3c69781812c7e, 7cd41621ab324b509ab8481e8a896b5e; no voters:
I20250904 19:57:53.228793 10857 raft_consensus.cc:2802] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:57:53.229436 10875 leader_election.cc:290] T cf72ca7d4cb4466fb0ed0305e7f82dba P 65dc647c385342abbec3c69781812c7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd0d980a231c4ccca173b477f06233d4 (127.10.24.129:37523), 7cd41621ab324b509ab8481e8a896b5e (127.10.24.130:37445)
I20250904 19:57:53.231248 10857 raft_consensus.cc:695] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [term 1 LEADER]: Becoming Leader. State: Replica: 7cd41621ab324b509ab8481e8a896b5e, State: Running, Role: LEADER
I20250904 19:57:53.232908 10637 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "cf72ca7d4cb4466fb0ed0305e7f82dba" candidate_uuid: "65dc647c385342abbec3c69781812c7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd0d980a231c4ccca173b477f06233d4" is_pre_election: true
I20250904 19:57:53.233750 10637 raft_consensus.cc:2391] T cf72ca7d4cb4466fb0ed0305e7f82dba P dd0d980a231c4ccca173b477f06233d4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 65dc647c385342abbec3c69781812c7e in current term 1: Already voted for candidate 7cd41621ab324b509ab8481e8a896b5e in this term.
I20250904 19:57:53.235023 10857 consensus_queue.cc:237] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } }
I20250904 19:57:53.240970 10854 leader_election.cc:290] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [CANDIDATE]: Term 1 election: Requested vote from peers dd0d980a231c4ccca173b477f06233d4 (127.10.24.129:37523), 65dc647c385342abbec3c69781812c7e (127.10.24.131:42491)
I20250904 19:57:53.243525 10716 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "cf72ca7d4cb4466fb0ed0305e7f82dba" candidate_uuid: "65dc647c385342abbec3c69781812c7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cd41621ab324b509ab8481e8a896b5e" is_pre_election: true
I20250904 19:57:53.244534 10638 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3d7958cd6f844eab8f4406cffa79be43" candidate_uuid: "7cd41621ab324b509ab8481e8a896b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd0d980a231c4ccca173b477f06233d4"
I20250904 19:57:53.251787 10758 leader_election.cc:304] T cf72ca7d4cb4466fb0ed0305e7f82dba P 65dc647c385342abbec3c69781812c7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 65dc647c385342abbec3c69781812c7e; no voters: 7cd41621ab324b509ab8481e8a896b5e, dd0d980a231c4ccca173b477f06233d4
I20250904 19:57:53.253230 10638 raft_consensus.cc:3058] T 3d7958cd6f844eab8f4406cffa79be43 P dd0d980a231c4ccca173b477f06233d4 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:53.255748 10874 raft_consensus.cc:3058] T cf72ca7d4cb4466fb0ed0305e7f82dba P 65dc647c385342abbec3c69781812c7e [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:53.269209 10874 raft_consensus.cc:2747] T cf72ca7d4cb4466fb0ed0305e7f82dba P 65dc647c385342abbec3c69781812c7e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250904 19:57:53.300879 10638 raft_consensus.cc:2466] T 3d7958cd6f844eab8f4406cffa79be43 P dd0d980a231c4ccca173b477f06233d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cd41621ab324b509ab8481e8a896b5e in term 1.
I20250904 19:57:53.317245 10367 catalog_manager.cc:5582] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cd41621ab324b509ab8481e8a896b5e (127.10.24.130). New cstate: current_term: 1 leader_uuid: "7cd41621ab324b509ab8481e8a896b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } health_report { overall_health: UNKNOWN } } }
I20250904 19:57:53.318993 10370 catalog_manager.cc:5582] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 65dc647c385342abbec3c69781812c7e (127.10.24.131). New cstate: current_term: 1 leader_uuid: "65dc647c385342abbec3c69781812c7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 } health_report { overall_health: HEALTHY } } }
I20250904 19:57:53.668965 10857 consensus_queue.cc:1035] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:57:53.676656 10857 consensus_queue.cc:1035] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:57:53.729174 10874 consensus_queue.cc:1035] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cd41621ab324b509ab8481e8a896b5e" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 37445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:57:53.752347 10874 consensus_queue.cc:1035] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:57:53.803805 10877 consensus_queue.cc:1035] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dc647c385342abbec3c69781812c7e" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 42491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:57:53.860805 10877 consensus_queue.cc:1035] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd0d980a231c4ccca173b477f06233d4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 37523 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250904 19:57:53.900825 10758 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.24.130:37445, user_credentials={real_user=slave}} blocked reactor thread for 69531us
I20250904 19:57:54.534158 10783 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250904 19:57:54.581447 10705 tablet_service.cc:1430] Tablet server has 2 leaders and 0 scanners
I20250904 19:57:54.628118 10627 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250904 19:57:54.663762 10679 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.24.129:37523, user_credentials={real_user=slave}} blocked reactor thread for 50483.2us
I20250904 19:57:55.001292 10338 tablet_server.cc:178] TabletServer@127.10.24.129:0 shutting down...
I20250904 19:57:55.054832 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:57:55.055586 10338 tablet_replica.cc:331] T 3d7958cd6f844eab8f4406cffa79be43 P dd0d980a231c4ccca173b477f06233d4: stopping tablet replica
I20250904 19:57:55.056277 10338 raft_consensus.cc:2241] T 3d7958cd6f844eab8f4406cffa79be43 P dd0d980a231c4ccca173b477f06233d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:57:55.056725 10338 raft_consensus.cc:2270] T 3d7958cd6f844eab8f4406cffa79be43 P dd0d980a231c4ccca173b477f06233d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:55.059386 10338 tablet_replica.cc:331] T ace90b31faeb4a63a2791ce38c5821c1 P dd0d980a231c4ccca173b477f06233d4: stopping tablet replica
I20250904 19:57:55.059865 10338 raft_consensus.cc:2241] T ace90b31faeb4a63a2791ce38c5821c1 P dd0d980a231c4ccca173b477f06233d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:57:55.060245 10338 raft_consensus.cc:2270] T ace90b31faeb4a63a2791ce38c5821c1 P dd0d980a231c4ccca173b477f06233d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:55.061764 10338 tablet_replica.cc:331] T cf72ca7d4cb4466fb0ed0305e7f82dba P dd0d980a231c4ccca173b477f06233d4: stopping tablet replica
I20250904 19:57:55.062175 10338 raft_consensus.cc:2241] T cf72ca7d4cb4466fb0ed0305e7f82dba P dd0d980a231c4ccca173b477f06233d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:57:55.062516 10338 raft_consensus.cc:2270] T cf72ca7d4cb4466fb0ed0305e7f82dba P dd0d980a231c4ccca173b477f06233d4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250904 19:57:55.247121 10679 consensus_peers.cc:489] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e -> Peer dd0d980a231c4ccca173b477f06233d4 (127.10.24.129:37523): Couldn't send request to peer dd0d980a231c4ccca173b477f06233d4. Status: Network error: Client connection negotiation failed: client connection to 127.10.24.129:37523: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 19:57:55.247913 10338 tablet_server.cc:195] TabletServer@127.10.24.129:0 shutdown complete.
I20250904 19:57:55.261251 10338 tablet_server.cc:178] TabletServer@127.10.24.130:0 shutting down...
W20250904 19:57:55.298637 10679 consensus_peers.cc:489] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e -> Peer dd0d980a231c4ccca173b477f06233d4 (127.10.24.129:37523): Couldn't send request to peer dd0d980a231c4ccca173b477f06233d4. Status: Network error: Client connection negotiation failed: client connection to 127.10.24.129:37523: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 19:57:55.301595 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:57:55.302146 10338 tablet_replica.cc:331] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e: stopping tablet replica
I20250904 19:57:55.302899 10338 raft_consensus.cc:2241] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [term 1 LEADER]: Raft consensus shutting down.
W20250904 19:57:55.303104 10757 consensus_peers.cc:489] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e -> Peer dd0d980a231c4ccca173b477f06233d4 (127.10.24.129:37523): Couldn't send request to peer dd0d980a231c4ccca173b477f06233d4. Status: Network error: Client connection negotiation failed: client connection to 127.10.24.129:37523: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 19:57:55.303752 10338 raft_consensus.cc:2270] T 3d7958cd6f844eab8f4406cffa79be43 P 7cd41621ab324b509ab8481e8a896b5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:55.305454 10338 tablet_replica.cc:331] T ace90b31faeb4a63a2791ce38c5821c1 P 7cd41621ab324b509ab8481e8a896b5e: stopping tablet replica
I20250904 19:57:55.305899 10338 raft_consensus.cc:2241] T ace90b31faeb4a63a2791ce38c5821c1 P 7cd41621ab324b509ab8481e8a896b5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:57:55.306279 10338 raft_consensus.cc:2270] T ace90b31faeb4a63a2791ce38c5821c1 P 7cd41621ab324b509ab8481e8a896b5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:55.308202 10338 tablet_replica.cc:331] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e: stopping tablet replica
I20250904 19:57:55.308712 10338 raft_consensus.cc:2241] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:57:55.309469 10338 raft_consensus.cc:2270] T cf72ca7d4cb4466fb0ed0305e7f82dba P 7cd41621ab324b509ab8481e8a896b5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:55.329250 10338 tablet_server.cc:195] TabletServer@127.10.24.130:0 shutdown complete.
I20250904 19:57:55.341504 10338 tablet_server.cc:178] TabletServer@127.10.24.131:0 shutting down...
I20250904 19:57:55.365935 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:57:55.366490 10338 tablet_replica.cc:331] T cf72ca7d4cb4466fb0ed0305e7f82dba P 65dc647c385342abbec3c69781812c7e: stopping tablet replica
I20250904 19:57:55.367041 10338 raft_consensus.cc:2241] T cf72ca7d4cb4466fb0ed0305e7f82dba P 65dc647c385342abbec3c69781812c7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:57:55.367413 10338 raft_consensus.cc:2270] T cf72ca7d4cb4466fb0ed0305e7f82dba P 65dc647c385342abbec3c69781812c7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:55.368916 10338 tablet_replica.cc:331] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e: stopping tablet replica
I20250904 19:57:55.369336 10338 raft_consensus.cc:2241] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:57:55.369935 10338 raft_consensus.cc:2270] T ace90b31faeb4a63a2791ce38c5821c1 P 65dc647c385342abbec3c69781812c7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:55.371402 10338 tablet_replica.cc:331] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e: stopping tablet replica
I20250904 19:57:55.371837 10338 raft_consensus.cc:2241] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:57:55.372169 10338 raft_consensus.cc:2270] T 3d7958cd6f844eab8f4406cffa79be43 P 65dc647c385342abbec3c69781812c7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:55.391387 10338 tablet_server.cc:195] TabletServer@127.10.24.131:0 shutdown complete.
I20250904 19:57:55.401846 10338 master.cc:561] Master@127.10.24.190:44751 shutting down...
I20250904 19:57:55.468540 10576 catalog_manager.cc:797] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Closing
I20250904 19:57:55.501789 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:57:55.502609 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:55.502948 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P da0e75bb907941f8946c895ced46a241: stopping tablet replica
I20250904 19:57:55.520828 10338 master.cc:583] Master@127.10.24.190:44751 shutdown complete.
I20250904 19:57:55.531298 10338 master.cc:561] Master@127.10.24.189:46147 shutting down...
I20250904 19:57:55.543898 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:57:55.544348 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:55.544579 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 650c62ea71eb46aaa624a6aadceb250c: stopping tablet replica
I20250904 19:57:55.561827 10338 master.cc:583] Master@127.10.24.189:46147 shutdown complete.
I20250904 19:57:55.571303 10338 master.cc:561] Master@127.10.24.188:37175 shutting down...
I20250904 19:57:55.584895 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:57:55.585397 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:55.585672 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P c19903b003444a96a9dbae99f4422ac3: stopping tablet replica
I20250904 19:57:55.603379 10338 master.cc:583] Master@127.10.24.188:37175 shutdown complete.
[ OK ] RemoteKsckTest.TestClusterOk (6054 ms)
[ RUN ] RemoteKsckTest.TestCheckUnusualFlags
I20250904 19:57:55.631240 10338 test_util.cc:276] Using random seed: 589412468
I20250904 19:57:55.632189 10338 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.24.190:36775,127.10.24.189:39855,127.10.24.188:41619
I20250904 19:57:55.633401 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:57:55.637797 10947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:57:55.638192 10948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:57:55.640093 10950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:55.640740 10338 server_base.cc:1047] running on GCE node
I20250904 19:57:55.641777 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:57:55.641953 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:57:55.642096 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015875642081 us; error 0 us; skew 500 ppm
I20250904 19:57:55.642688 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:57:55.645083 10338 webserver.cc:480] Webserver started at http://127.10.24.190:38095/ using document root <none> and password file <none>
I20250904 19:57:55.645504 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:57:55.645663 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:57:55.645915 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:57:55.646835 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/master-0-root/instance:
uuid: "8092d6d8791340cf9e5e9c08d962e0c2"
format_stamp: "Formatted at 2025-09-04 19:57:55 on dist-test-slave-f3xj"
I20250904 19:57:55.650653 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20250904 19:57:55.653512 10955 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:55.654242 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20250904 19:57:55.654474 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/master-0-root
uuid: "8092d6d8791340cf9e5e9c08d962e0c2"
format_stamp: "Formatted at 2025-09-04 19:57:55 on dist-test-slave-f3xj"
I20250904 19:57:55.654695 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:57:55.665601 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:57:55.666481 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:57:55.698329 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.190:36775
I20250904 19:57:55.698421 11006 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.190:36775 every 8 connection(s)
I20250904 19:57:55.701874 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:57:55.702013 11007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20250904 19:57:55.706738 11009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:57:55.708472 11010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:57:55.709149 11012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:55.709035 11007 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:57:55.709789 10338 server_base.cc:1047] running on GCE node
I20250904 19:57:55.710654 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:57:55.710853 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:57:55.711038 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015875711019 us; error 0 us; skew 500 ppm
I20250904 19:57:55.711733 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:57:55.714288 10338 webserver.cc:480] Webserver started at http://127.10.24.189:46505/ using document root <none> and password file <none>
I20250904 19:57:55.714896 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:57:55.715123 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:57:55.715355 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:57:55.716287 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/master-1-root/instance:
uuid: "1fef062f07b7422098d5a5d189809b6e"
format_stamp: "Formatted at 2025-09-04 19:57:55 on dist-test-slave-f3xj"
I20250904 19:57:55.721488 11007 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:57:55.721619 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20250904 19:57:55.725389 11020 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:55.726131 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20250904 19:57:55.726379 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/master-1-root
uuid: "1fef062f07b7422098d5a5d189809b6e"
format_stamp: "Formatted at 2025-09-04 19:57:55 on dist-test-slave-f3xj"
W20250904 19:57:55.726580 11007 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.189:39855: Network error: Client connection negotiation failed: client connection to 127.10.24.189:39855: connect: Connection refused (error 111)
I20250904 19:57:55.726711 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:57:55.741545 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:57:55.742482 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:57:55.745070 11007 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } attempt: 1
W20250904 19:57:55.748777 11007 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.189:39855: Network error: Client connection negotiation failed: client connection to 127.10.24.189:39855: connect: Connection refused (error 111)
I20250904 19:57:55.776110 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.189:39855
I20250904 19:57:55.776206 11071 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.189:39855 every 8 connection(s)
I20250904 19:57:55.779667 11072 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:55.779863 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:57:55.784991 11074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:57:55.785540 11075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:55.787297 11072 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:57:55.788163 10338 server_base.cc:1047] running on GCE node
W20250904 19:57:55.789497 11077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:55.790417 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:57:55.790676 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:57:55.790879 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015875790857 us; error 0 us; skew 500 ppm
I20250904 19:57:55.791503 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:57:55.793931 10338 webserver.cc:480] Webserver started at http://127.10.24.188:42781/ using document root <none> and password file <none>
I20250904 19:57:55.794523 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:57:55.794752 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:57:55.795053 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:57:55.796372 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/master-2-root/instance:
uuid: "4179ea0edfa5437ba65276fe8add60f2"
format_stamp: "Formatted at 2025-09-04 19:57:55 on dist-test-slave-f3xj"
I20250904 19:57:55.798697 11072 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:57:55.801577 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20250904 19:57:55.805207 11084 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:55.806293 10338 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20250904 19:57:55.806635 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/master-2-root
uuid: "4179ea0edfa5437ba65276fe8add60f2"
format_stamp: "Formatted at 2025-09-04 19:57:55 on dist-test-slave-f3xj"
I20250904 19:57:55.806974 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:57:55.814860 11072 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:57:55.819622 11072 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:41619: Network error: Client connection negotiation failed: client connection to 127.10.24.188:41619: connect: Connection refused (error 111)
I20250904 19:57:55.821158 11007 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } attempt: 2
I20250904 19:57:55.825212 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:57:55.826535 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:57:55.830653 11007 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:57:55.834410 11007 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:41619: Network error: Client connection negotiation failed: client connection to 127.10.24.188:41619: connect: Connection refused (error 111)
I20250904 19:57:55.861510 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.188:41619
I20250904 19:57:55.861599 11136 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.188:41619 every 8 connection(s)
I20250904 19:57:55.864547 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 19:57:55.865243 11137 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:55.870462 11137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:57:55.880249 11137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:57:55.885046 11072 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } attempt: 1
I20250904 19:57:55.893858 11137 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:57:55.896814 11007 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } attempt: 1
I20250904 19:57:55.901453 11072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e: Bootstrap starting.
I20250904 19:57:55.906706 11072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:55.909976 11137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2: Bootstrap starting.
I20250904 19:57:55.913534 11072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e: No bootstrap required, opened a new log
I20250904 19:57:55.914775 11007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2: Bootstrap starting.
I20250904 19:57:55.915781 11137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:55.915623 11072 raft_consensus.cc:357] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } } peers { permanent_uuid: "1fef062f07b7422098d5a5d189809b6e" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } } peers { permanent_uuid: "4179ea0edfa5437ba65276fe8add60f2" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } }
I20250904 19:57:55.916203 11072 raft_consensus.cc:383] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:55.916432 11072 raft_consensus.cc:738] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fef062f07b7422098d5a5d189809b6e, State: Initialized, Role: FOLLOWER
I20250904 19:57:55.916986 11072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } } peers { permanent_uuid: "1fef062f07b7422098d5a5d189809b6e" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } } peers { permanent_uuid: "4179ea0edfa5437ba65276fe8add60f2" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } }
I20250904 19:57:55.918774 11143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } } peers { permanent_uuid: "1fef062f07b7422098d5a5d189809b6e" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } } peers { permanent_uuid: "4179ea0edfa5437ba65276fe8add60f2" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } } }
I20250904 19:57:55.919601 11143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:55.920462 11072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:57:55.920743 11007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:55.922339 11137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2: No bootstrap required, opened a new log
I20250904 19:57:55.925172 11137 raft_consensus.cc:357] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } } peers { permanent_uuid: "1fef062f07b7422098d5a5d189809b6e" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } } peers { permanent_uuid: "4179ea0edfa5437ba65276fe8add60f2" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } }
I20250904 19:57:55.925658 11007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2: No bootstrap required, opened a new log
I20250904 19:57:55.925976 11137 raft_consensus.cc:383] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:55.926321 11137 raft_consensus.cc:738] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4179ea0edfa5437ba65276fe8add60f2, State: Initialized, Role: FOLLOWER
I20250904 19:57:55.927069 11137 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } } peers { permanent_uuid: "1fef062f07b7422098d5a5d189809b6e" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } } peers { permanent_uuid: "4179ea0edfa5437ba65276fe8add60f2" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } }
I20250904 19:57:55.928287 11007 raft_consensus.cc:357] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } } peers { permanent_uuid: "1fef062f07b7422098d5a5d189809b6e" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } } peers { permanent_uuid: "4179ea0edfa5437ba65276fe8add60f2" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } }
I20250904 19:57:55.928949 11007 raft_consensus.cc:383] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:55.929253 11007 raft_consensus.cc:738] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8092d6d8791340cf9e5e9c08d962e0c2, State: Initialized, Role: FOLLOWER
I20250904 19:57:55.929605 11152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } } peers { permanent_uuid: "1fef062f07b7422098d5a5d189809b6e" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } } peers { permanent_uuid: "4179ea0edfa5437ba65276fe8add60f2" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } } }
I20250904 19:57:55.930387 11152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:55.930012 11007 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } } peers { permanent_uuid: "1fef062f07b7422098d5a5d189809b6e" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } } peers { permanent_uuid: "4179ea0edfa5437ba65276fe8add60f2" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } }
I20250904 19:57:55.931538 11137 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:57:55.932134 11154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } } peers { permanent_uuid: "1fef062f07b7422098d5a5d189809b6e" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } } peers { permanent_uuid: "4179ea0edfa5437ba65276fe8add60f2" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } } }
I20250904 19:57:55.932890 11154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:55.933712 11007 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:57:55.939172 11154 raft_consensus.cc:491] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:57:55.939692 11154 raft_consensus.cc:513] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } } peers { permanent_uuid: "1fef062f07b7422098d5a5d189809b6e" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } } peers { permanent_uuid: "4179ea0edfa5437ba65276fe8add60f2" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } }
I20250904 19:57:55.941802 11154 leader_election.cc:290] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1fef062f07b7422098d5a5d189809b6e (127.10.24.189:39855), 4179ea0edfa5437ba65276fe8add60f2 (127.10.24.188:41619)
I20250904 19:57:55.943925 11047 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fef062f07b7422098d5a5d189809b6e" is_pre_election: true
I20250904 19:57:55.944384 11112 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4179ea0edfa5437ba65276fe8add60f2" is_pre_election: true
I20250904 19:57:55.944589 11047 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8092d6d8791340cf9e5e9c08d962e0c2 in term 0.
I20250904 19:57:55.945000 11112 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8092d6d8791340cf9e5e9c08d962e0c2 in term 0.
I20250904 19:57:55.945842 10959 leader_election.cc:304] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fef062f07b7422098d5a5d189809b6e, 8092d6d8791340cf9e5e9c08d962e0c2; no voters:
I20250904 19:57:55.946780 11154 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:57:55.947052 11154 raft_consensus.cc:491] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:57:55.947299 11154 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:55.952126 11154 raft_consensus.cc:513] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } } peers { permanent_uuid: "1fef062f07b7422098d5a5d189809b6e" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } } peers { permanent_uuid: "4179ea0edfa5437ba65276fe8add60f2" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } }
W20250904 19:57:55.954480 11165 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20250904 19:57:55.954562 11047 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fef062f07b7422098d5a5d189809b6e"
W20250904 19:57:55.954923 11165 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250904 19:57:55.955178 11047 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:55.958047 11112 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4179ea0edfa5437ba65276fe8add60f2"
I20250904 19:57:55.958595 11112 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:55.961468 11047 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8092d6d8791340cf9e5e9c08d962e0c2 in term 1.
I20250904 19:57:55.962535 10959 leader_election.cc:304] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fef062f07b7422098d5a5d189809b6e, 8092d6d8791340cf9e5e9c08d962e0c2; no voters:
I20250904 19:57:55.964385 11174 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:57:55.964682 11112 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8092d6d8791340cf9e5e9c08d962e0c2 in term 1.
I20250904 19:57:55.965917 11154 leader_election.cc:290] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [CANDIDATE]: Term 1 election: Requested vote from peers 1fef062f07b7422098d5a5d189809b6e (127.10.24.189:39855), 4179ea0edfa5437ba65276fe8add60f2 (127.10.24.188:41619)
I20250904 19:57:55.967101 11174 raft_consensus.cc:695] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [term 1 LEADER]: Becoming Leader. State: Replica: 8092d6d8791340cf9e5e9c08d962e0c2, State: Running, Role: LEADER
I20250904 19:57:55.968102 11174 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } } peers { permanent_uuid: "1fef062f07b7422098d5a5d189809b6e" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } } peers { permanent_uuid: "4179ea0edfa5437ba65276fe8add60f2" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } }
W20250904 19:57:55.970713 11176 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:57:55.970968 11176 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250904 19:57:55.974133 11154 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8092d6d8791340cf9e5e9c08d962e0c2. Latest consensus state: current_term: 1 leader_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } } peers { permanent_uuid: "1fef062f07b7422098d5a5d189809b6e" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } } peers { permanent_uuid: "4179ea0edfa5437ba65276fe8add60f2" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } } }
I20250904 19:57:55.974637 11154 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [sys.catalog]: This master's current role is: LEADER
I20250904 19:57:55.976316 11181 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 19:57:55.980481 11181 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 19:57:55.981456 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20250904 19:57:55.981722 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20250904 19:57:55.982913 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:57:55.988643 11112 raft_consensus.cc:1273] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2 [term 1 FOLLOWER]: Refusing update from remote peer 8092d6d8791340cf9e5e9c08d962e0c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:57:55.989080 11047 raft_consensus.cc:1273] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e [term 1 FOLLOWER]: Refusing update from remote peer 8092d6d8791340cf9e5e9c08d962e0c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20250904 19:57:55.990176 11186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:55.990465 11154 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4179ea0edfa5437ba65276fe8add60f2" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250904 19:57:55.991189 11174 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fef062f07b7422098d5a5d189809b6e" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250904 19:57:55.991482 11187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:56.003002 10338 server_base.cc:1047] running on GCE node
W20250904 19:57:56.004367 11189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:56.005378 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:57:56.005637 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:57:56.005887 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015876005865 us; error 0 us; skew 500 ppm
I20250904 19:57:56.005698 11152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8092d6d8791340cf9e5e9c08d962e0c2. Latest consensus state: current_term: 1 leader_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } } peers { permanent_uuid: "1fef062f07b7422098d5a5d189809b6e" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } } peers { permanent_uuid: "4179ea0edfa5437ba65276fe8add60f2" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } } }
I20250904 19:57:56.006525 11143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8092d6d8791340cf9e5e9c08d962e0c2. Latest consensus state: current_term: 1 leader_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } } peers { permanent_uuid: "1fef062f07b7422098d5a5d189809b6e" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } } peers { permanent_uuid: "4179ea0edfa5437ba65276fe8add60f2" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } } }
I20250904 19:57:56.006574 11152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:56.007279 11143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:56.006996 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:57:56.009788 11185 mvcc.cc:204] Tried to move back new op lower bound from 7196737028036907008 to 7196737027974828032. Current Snapshot: MvccSnapshot[applied={T|T < 7196737028036907008}]
I20250904 19:57:56.014153 11175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } } peers { permanent_uuid: "1fef062f07b7422098d5a5d189809b6e" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } } peers { permanent_uuid: "4179ea0edfa5437ba65276fe8add60f2" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } } }
I20250904 19:57:56.014873 11175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [sys.catalog]: This master's current role is: LEADER
I20250904 19:57:56.015513 11175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } } peers { permanent_uuid: "1fef062f07b7422098d5a5d189809b6e" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } } peers { permanent_uuid: "4179ea0edfa5437ba65276fe8add60f2" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } } }
I20250904 19:57:56.016234 11175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [sys.catalog]: This master's current role is: LEADER
I20250904 19:57:56.017254 11143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } } peers { permanent_uuid: "1fef062f07b7422098d5a5d189809b6e" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } } peers { permanent_uuid: "4179ea0edfa5437ba65276fe8add60f2" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } } }
I20250904 19:57:56.017949 11143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:56.018078 11181 catalog_manager.cc:1349] Generated new cluster ID: 817d1dc0b3834a5ea900a31dfdfef1f1
I20250904 19:57:56.018378 11181 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 19:57:56.019390 10338 webserver.cc:480] Webserver started at http://127.10.24.129:35727/ using document root <none> and password file <none>
I20250904 19:57:56.020074 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:57:56.020313 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:57:56.020627 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:57:56.021971 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/ts-0-root/instance:
uuid: "59b040f0f42347a39a014d1a9a83eb22"
format_stamp: "Formatted at 2025-09-04 19:57:56 on dist-test-slave-f3xj"
I20250904 19:57:56.022068 11152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8092d6d8791340cf9e5e9c08d962e0c2" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 36775 } } peers { permanent_uuid: "1fef062f07b7422098d5a5d189809b6e" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 39855 } } peers { permanent_uuid: "4179ea0edfa5437ba65276fe8add60f2" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41619 } } }
I20250904 19:57:56.022732 11152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:56.027423 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20250904 19:57:56.031765 11201 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:56.032560 10338 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20250904 19:57:56.032881 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/ts-0-root
uuid: "59b040f0f42347a39a014d1a9a83eb22"
format_stamp: "Formatted at 2025-09-04 19:57:56 on dist-test-slave-f3xj"
I20250904 19:57:56.033223 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:57:56.044960 11181 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 19:57:56.046730 11181 catalog_manager.cc:1506] Loading token signing keys...
I20250904 19:57:56.060364 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:57:56.061757 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:57:56.063791 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:57:56.066952 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:57:56.067210 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:56.067488 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:57:56.067706 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20250904 19:57:56.071074 11181 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2: Generated new TSK 0
I20250904 19:57:56.071758 11181 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 19:57:56.109580 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.129:36707
I20250904 19:57:56.109670 11263 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.129:36707 every 8 connection(s)
I20250904 19:57:56.127393 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:57:56.127707 11264 heartbeater.cc:344] Connected to a master server at 127.10.24.189:39855
I20250904 19:57:56.128188 11264 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:56.129079 11264 heartbeater.cc:507] Master 127.10.24.189:39855 requested a full tablet report, sending...
I20250904 19:57:56.132097 11037 ts_manager.cc:194] Registered new tserver with Master: 59b040f0f42347a39a014d1a9a83eb22 (127.10.24.129:36707)
I20250904 19:57:56.136060 11265 heartbeater.cc:344] Connected to a master server at 127.10.24.188:41619
I20250904 19:57:56.136498 11265 heartbeater.cc:461] Registering TS with master...
W20250904 19:57:56.137132 11271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:56.137578 11265 heartbeater.cc:507] Master 127.10.24.188:41619 requested a full tablet report, sending...
W20250904 19:57:56.138376 11272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:56.140758 11102 ts_manager.cc:194] Registered new tserver with Master: 59b040f0f42347a39a014d1a9a83eb22 (127.10.24.129:36707)
I20250904 19:57:56.142612 11266 heartbeater.cc:344] Connected to a master server at 127.10.24.190:36775
I20250904 19:57:56.142997 11266 heartbeater.cc:461] Registering TS with master...
W20250904 19:57:56.143715 11274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:56.143913 11266 heartbeater.cc:507] Master 127.10.24.190:36775 requested a full tablet report, sending...
I20250904 19:57:56.143899 10338 server_base.cc:1047] running on GCE node
I20250904 19:57:56.145120 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:57:56.145363 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:57:56.145567 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015876145550 us; error 0 us; skew 500 ppm
I20250904 19:57:56.146144 10972 ts_manager.cc:194] Registered new tserver with Master: 59b040f0f42347a39a014d1a9a83eb22 (127.10.24.129:36707)
I20250904 19:57:56.146250 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:57:56.148074 10972 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37456
I20250904 19:57:56.148815 10338 webserver.cc:480] Webserver started at http://127.10.24.130:44025/ using document root <none> and password file <none>
I20250904 19:57:56.149296 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:57:56.149461 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:57:56.149698 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:57:56.150647 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/ts-1-root/instance:
uuid: "a3196a79eebf44a9b182b1b92b75fa91"
format_stamp: "Formatted at 2025-09-04 19:57:56 on dist-test-slave-f3xj"
I20250904 19:57:56.154935 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20250904 19:57:56.157915 11279 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:56.158728 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250904 19:57:56.159060 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/ts-1-root
uuid: "a3196a79eebf44a9b182b1b92b75fa91"
format_stamp: "Formatted at 2025-09-04 19:57:56 on dist-test-slave-f3xj"
I20250904 19:57:56.159394 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:57:56.160102 11266 heartbeater.cc:499] Master 127.10.24.190:36775 was elected leader, sending a full tablet report...
I20250904 19:57:56.206275 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:57:56.207425 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:57:56.208963 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:57:56.211454 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:57:56.211656 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:56.211889 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:57:56.212025 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:56.256160 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.130:45747
I20250904 19:57:56.256263 11341 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.130:45747 every 8 connection(s)
I20250904 19:57:56.264467 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:57:56.275049 11349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:56.277413 11342 heartbeater.cc:344] Connected to a master server at 127.10.24.189:39855
I20250904 19:57:56.277877 11342 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:56.278685 11342 heartbeater.cc:507] Master 127.10.24.189:39855 requested a full tablet report, sending...
I20250904 19:57:56.282629 11037 ts_manager.cc:194] Registered new tserver with Master: a3196a79eebf44a9b182b1b92b75fa91 (127.10.24.130:45747)
W20250904 19:57:56.283265 11350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:56.284225 11343 heartbeater.cc:344] Connected to a master server at 127.10.24.188:41619
I20250904 19:57:56.284660 11343 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:56.285578 11343 heartbeater.cc:507] Master 127.10.24.188:41619 requested a full tablet report, sending...
I20250904 19:57:56.285741 11344 heartbeater.cc:344] Connected to a master server at 127.10.24.190:36775
I20250904 19:57:56.286446 11344 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:56.287268 11344 heartbeater.cc:507] Master 127.10.24.190:36775 requested a full tablet report, sending...
I20250904 19:57:56.289538 10972 ts_manager.cc:194] Registered new tserver with Master: a3196a79eebf44a9b182b1b92b75fa91 (127.10.24.130:45747)
I20250904 19:57:56.290192 11101 ts_manager.cc:194] Registered new tserver with Master: a3196a79eebf44a9b182b1b92b75fa91 (127.10.24.130:45747)
I20250904 19:57:56.291306 10972 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37470
W20250904 19:57:56.294606 11352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:56.294766 10338 server_base.cc:1047] running on GCE node
I20250904 19:57:56.295688 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:57:56.295948 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:57:56.296137 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015876296120 us; error 0 us; skew 500 ppm
I20250904 19:57:56.296823 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:57:56.299156 10338 webserver.cc:480] Webserver started at http://127.10.24.131:45151/ using document root <none> and password file <none>
I20250904 19:57:56.299618 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:57:56.299799 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:57:56.300036 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:57:56.300974 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/ts-2-root/instance:
uuid: "95b4a22c401f42148d5a6e4dde77a2f0"
format_stamp: "Formatted at 2025-09-04 19:57:56 on dist-test-slave-f3xj"
I20250904 19:57:56.304076 11344 heartbeater.cc:499] Master 127.10.24.190:36775 was elected leader, sending a full tablet report...
I20250904 19:57:56.305943 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20250904 19:57:56.311297 11357 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:56.311968 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250904 19:57:56.312248 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/ts-2-root
uuid: "95b4a22c401f42148d5a6e4dde77a2f0"
format_stamp: "Formatted at 2025-09-04 19:57:56 on dist-test-slave-f3xj"
I20250904 19:57:56.312486 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestCheckUnusualFlags.1757015869512910-10338-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:57:56.337291 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:57:56.338533 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:57:56.339869 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:57:56.341946 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:57:56.342130 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:56.342306 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:57:56.342428 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:56.398873 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.131:43909
I20250904 19:57:56.398969 11419 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.131:43909 every 8 connection(s)
I20250904 19:57:56.510190 11420 heartbeater.cc:344] Connected to a master server at 127.10.24.189:39855
I20250904 19:57:56.510646 11420 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:56.511549 11420 heartbeater.cc:507] Master 127.10.24.189:39855 requested a full tablet report, sending...
I20250904 19:57:56.514272 11036 ts_manager.cc:194] Registered new tserver with Master: 95b4a22c401f42148d5a6e4dde77a2f0 (127.10.24.131:43909)
I20250904 19:57:56.515950 11421 heartbeater.cc:344] Connected to a master server at 127.10.24.188:41619
I20250904 19:57:56.516346 11421 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:56.517196 11421 heartbeater.cc:507] Master 127.10.24.188:41619 requested a full tablet report, sending...
I20250904 19:57:56.521880 11101 ts_manager.cc:194] Registered new tserver with Master: 95b4a22c401f42148d5a6e4dde77a2f0 (127.10.24.131:43909)
I20250904 19:57:56.523862 11422 heartbeater.cc:344] Connected to a master server at 127.10.24.190:36775
I20250904 19:57:56.524150 11422 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:56.524780 11422 heartbeater.cc:507] Master 127.10.24.190:36775 requested a full tablet report, sending...
I20250904 19:57:56.526605 10972 ts_manager.cc:194] Registered new tserver with Master: 95b4a22c401f42148d5a6e4dde77a2f0 (127.10.24.131:43909)
I20250904 19:57:56.527514 10338 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.041870224s
I20250904 19:57:56.528301 10972 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37474
I20250904 19:57:56.540612 11422 heartbeater.cc:499] Master 127.10.24.190:36775 was elected leader, sending a full tablet report...
I20250904 19:57:56.569555 10971 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37478:
name: "ksck-test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\n\000\000\000\004\001\024\000\000\000""\004\001\n\000\000\000\004\001\024\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 19:57:56.572006 10971 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table ksck-test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 19:57:56.708784 11229 tablet_service.cc:1468] Processing CreateTablet for tablet 393ff9b87b4146f28b89225252e5277c (DEFAULT_TABLE table=ksck-test-table [id=8b50f9acced0466abe42fcc452f3faf1]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:57:56.709978 11229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 393ff9b87b4146f28b89225252e5277c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:56.713793 11228 tablet_service.cc:1468] Processing CreateTablet for tablet 6da3727db70e4184a3f8751ae7b5fe17 (DEFAULT_TABLE table=ksck-test-table [id=8b50f9acced0466abe42fcc452f3faf1]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:57:56.715137 11228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6da3727db70e4184a3f8751ae7b5fe17. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:56.723709 11227 tablet_service.cc:1468] Processing CreateTablet for tablet 9554457d24c044c78927500c90be27dc (DEFAULT_TABLE table=ksck-test-table [id=8b50f9acced0466abe42fcc452f3faf1]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:57:56.724987 11227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9554457d24c044c78927500c90be27dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:56.724972 11385 tablet_service.cc:1468] Processing CreateTablet for tablet 393ff9b87b4146f28b89225252e5277c (DEFAULT_TABLE table=ksck-test-table [id=8b50f9acced0466abe42fcc452f3faf1]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:57:56.726253 11385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 393ff9b87b4146f28b89225252e5277c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:56.740391 11384 tablet_service.cc:1468] Processing CreateTablet for tablet 6da3727db70e4184a3f8751ae7b5fe17 (DEFAULT_TABLE table=ksck-test-table [id=8b50f9acced0466abe42fcc452f3faf1]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:57:56.741695 11384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6da3727db70e4184a3f8751ae7b5fe17. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:56.752286 11448 tablet_bootstrap.cc:492] T 6da3727db70e4184a3f8751ae7b5fe17 P 59b040f0f42347a39a014d1a9a83eb22: Bootstrap starting.
I20250904 19:57:56.756747 11449 tablet_bootstrap.cc:492] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0: Bootstrap starting.
I20250904 19:57:56.760592 11385 tablet_service.cc:1468] Processing CreateTablet for tablet 9554457d24c044c78927500c90be27dc (DEFAULT_TABLE table=ksck-test-table [id=8b50f9acced0466abe42fcc452f3faf1]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:57:56.763715 11448 tablet_bootstrap.cc:654] T 6da3727db70e4184a3f8751ae7b5fe17 P 59b040f0f42347a39a014d1a9a83eb22: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:56.765327 11385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9554457d24c044c78927500c90be27dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:56.770181 11307 tablet_service.cc:1468] Processing CreateTablet for tablet 393ff9b87b4146f28b89225252e5277c (DEFAULT_TABLE table=ksck-test-table [id=8b50f9acced0466abe42fcc452f3faf1]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:57:56.771447 11307 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 393ff9b87b4146f28b89225252e5277c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:56.776511 11449 tablet_bootstrap.cc:654] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:56.790287 11306 tablet_service.cc:1468] Processing CreateTablet for tablet 6da3727db70e4184a3f8751ae7b5fe17 (DEFAULT_TABLE table=ksck-test-table [id=8b50f9acced0466abe42fcc452f3faf1]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:57:56.791013 11305 tablet_service.cc:1468] Processing CreateTablet for tablet 9554457d24c044c78927500c90be27dc (DEFAULT_TABLE table=ksck-test-table [id=8b50f9acced0466abe42fcc452f3faf1]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:57:56.791544 11306 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6da3727db70e4184a3f8751ae7b5fe17. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:56.793144 11305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9554457d24c044c78927500c90be27dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:56.825562 11449 tablet_bootstrap.cc:492] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0: No bootstrap required, opened a new log
I20250904 19:57:56.826256 11449 ts_tablet_manager.cc:1397] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0: Time spent bootstrapping tablet: real 0.070s user 0.012s sys 0.004s
I20250904 19:57:56.834111 11452 tablet_bootstrap.cc:492] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91: Bootstrap starting.
I20250904 19:57:56.839856 11449 raft_consensus.cc:357] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } }
I20250904 19:57:56.840739 11449 raft_consensus.cc:383] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:56.841122 11449 raft_consensus.cc:738] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95b4a22c401f42148d5a6e4dde77a2f0, State: Initialized, Role: FOLLOWER
I20250904 19:57:56.843448 11452 tablet_bootstrap.cc:654] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:56.842010 11449 consensus_queue.cc:260] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } }
I20250904 19:57:56.857801 11452 tablet_bootstrap.cc:492] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91: No bootstrap required, opened a new log
I20250904 19:57:56.858276 11452 ts_tablet_manager.cc:1397] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91: Time spent bootstrapping tablet: real 0.024s user 0.011s sys 0.003s
I20250904 19:57:56.860997 11452 raft_consensus.cc:357] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } }
I20250904 19:57:56.866830 11452 raft_consensus.cc:383] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:56.867133 11452 raft_consensus.cc:738] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3196a79eebf44a9b182b1b92b75fa91, State: Initialized, Role: FOLLOWER
I20250904 19:57:56.868137 11448 tablet_bootstrap.cc:492] T 6da3727db70e4184a3f8751ae7b5fe17 P 59b040f0f42347a39a014d1a9a83eb22: No bootstrap required, opened a new log
I20250904 19:57:56.867862 11452 consensus_queue.cc:260] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } }
I20250904 19:57:56.868587 11448 ts_tablet_manager.cc:1397] T 6da3727db70e4184a3f8751ae7b5fe17 P 59b040f0f42347a39a014d1a9a83eb22: Time spent bootstrapping tablet: real 0.117s user 0.035s sys 0.046s
I20250904 19:57:56.872644 11448 raft_consensus.cc:357] T 6da3727db70e4184a3f8751ae7b5fe17 P 59b040f0f42347a39a014d1a9a83eb22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } }
I20250904 19:57:56.873283 11448 raft_consensus.cc:383] T 6da3727db70e4184a3f8751ae7b5fe17 P 59b040f0f42347a39a014d1a9a83eb22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:56.874496 11448 raft_consensus.cc:738] T 6da3727db70e4184a3f8751ae7b5fe17 P 59b040f0f42347a39a014d1a9a83eb22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59b040f0f42347a39a014d1a9a83eb22, State: Initialized, Role: FOLLOWER
I20250904 19:57:56.875241 11448 consensus_queue.cc:260] T 6da3727db70e4184a3f8751ae7b5fe17 P 59b040f0f42347a39a014d1a9a83eb22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } }
I20250904 19:57:56.884590 11448 ts_tablet_manager.cc:1428] T 6da3727db70e4184a3f8751ae7b5fe17 P 59b040f0f42347a39a014d1a9a83eb22: Time spent starting tablet: real 0.016s user 0.008s sys 0.004s
I20250904 19:57:56.888924 11448 tablet_bootstrap.cc:492] T 393ff9b87b4146f28b89225252e5277c P 59b040f0f42347a39a014d1a9a83eb22: Bootstrap starting.
I20250904 19:57:56.894340 11448 tablet_bootstrap.cc:654] T 393ff9b87b4146f28b89225252e5277c P 59b040f0f42347a39a014d1a9a83eb22: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:56.899360 11448 tablet_bootstrap.cc:492] T 393ff9b87b4146f28b89225252e5277c P 59b040f0f42347a39a014d1a9a83eb22: No bootstrap required, opened a new log
I20250904 19:57:56.899848 11448 ts_tablet_manager.cc:1397] T 393ff9b87b4146f28b89225252e5277c P 59b040f0f42347a39a014d1a9a83eb22: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20250904 19:57:56.902276 11448 raft_consensus.cc:357] T 393ff9b87b4146f28b89225252e5277c P 59b040f0f42347a39a014d1a9a83eb22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } }
I20250904 19:57:56.902921 11448 raft_consensus.cc:383] T 393ff9b87b4146f28b89225252e5277c P 59b040f0f42347a39a014d1a9a83eb22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:56.905174 11448 raft_consensus.cc:738] T 393ff9b87b4146f28b89225252e5277c P 59b040f0f42347a39a014d1a9a83eb22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59b040f0f42347a39a014d1a9a83eb22, State: Initialized, Role: FOLLOWER
I20250904 19:57:56.905875 11448 consensus_queue.cc:260] T 393ff9b87b4146f28b89225252e5277c P 59b040f0f42347a39a014d1a9a83eb22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } }
I20250904 19:57:56.908347 11452 ts_tablet_manager.cc:1428] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91: Time spent starting tablet: real 0.050s user 0.020s sys 0.024s
I20250904 19:57:56.909421 11452 tablet_bootstrap.cc:492] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91: Bootstrap starting.
I20250904 19:57:56.910933 11448 ts_tablet_manager.cc:1428] T 393ff9b87b4146f28b89225252e5277c P 59b040f0f42347a39a014d1a9a83eb22: Time spent starting tablet: real 0.011s user 0.004s sys 0.001s
I20250904 19:57:56.911947 11448 tablet_bootstrap.cc:492] T 9554457d24c044c78927500c90be27dc P 59b040f0f42347a39a014d1a9a83eb22: Bootstrap starting.
I20250904 19:57:56.915009 11452 tablet_bootstrap.cc:654] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:56.917536 11448 tablet_bootstrap.cc:654] T 9554457d24c044c78927500c90be27dc P 59b040f0f42347a39a014d1a9a83eb22: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:56.918701 11449 ts_tablet_manager.cc:1428] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0: Time spent starting tablet: real 0.092s user 0.009s sys 0.004s
I20250904 19:57:56.920385 11449 tablet_bootstrap.cc:492] T 9554457d24c044c78927500c90be27dc P 95b4a22c401f42148d5a6e4dde77a2f0: Bootstrap starting.
I20250904 19:57:56.925837 11449 tablet_bootstrap.cc:654] T 9554457d24c044c78927500c90be27dc P 95b4a22c401f42148d5a6e4dde77a2f0: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:56.942402 11448 tablet_bootstrap.cc:492] T 9554457d24c044c78927500c90be27dc P 59b040f0f42347a39a014d1a9a83eb22: No bootstrap required, opened a new log
I20250904 19:57:56.942812 11448 ts_tablet_manager.cc:1397] T 9554457d24c044c78927500c90be27dc P 59b040f0f42347a39a014d1a9a83eb22: Time spent bootstrapping tablet: real 0.031s user 0.016s sys 0.003s
I20250904 19:57:56.943377 11449 tablet_bootstrap.cc:492] T 9554457d24c044c78927500c90be27dc P 95b4a22c401f42148d5a6e4dde77a2f0: No bootstrap required, opened a new log
I20250904 19:57:56.944284 11449 ts_tablet_manager.cc:1397] T 9554457d24c044c78927500c90be27dc P 95b4a22c401f42148d5a6e4dde77a2f0: Time spent bootstrapping tablet: real 0.025s user 0.009s sys 0.003s
I20250904 19:57:56.945269 11448 raft_consensus.cc:357] T 9554457d24c044c78927500c90be27dc P 59b040f0f42347a39a014d1a9a83eb22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } }
I20250904 19:57:56.946056 11448 raft_consensus.cc:383] T 9554457d24c044c78927500c90be27dc P 59b040f0f42347a39a014d1a9a83eb22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:56.946336 11448 raft_consensus.cc:738] T 9554457d24c044c78927500c90be27dc P 59b040f0f42347a39a014d1a9a83eb22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59b040f0f42347a39a014d1a9a83eb22, State: Initialized, Role: FOLLOWER
I20250904 19:57:56.947117 11448 consensus_queue.cc:260] T 9554457d24c044c78927500c90be27dc P 59b040f0f42347a39a014d1a9a83eb22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } }
I20250904 19:57:56.949301 11448 ts_tablet_manager.cc:1428] T 9554457d24c044c78927500c90be27dc P 59b040f0f42347a39a014d1a9a83eb22: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20250904 19:57:56.960098 11165 catalog_manager.cc:1261] Loaded cluster ID: 817d1dc0b3834a5ea900a31dfdfef1f1
I20250904 19:57:56.960572 11165 catalog_manager.cc:1554] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e: loading cluster ID for follower catalog manager: success
I20250904 19:57:56.975529 11176 catalog_manager.cc:1261] Loaded cluster ID: 817d1dc0b3834a5ea900a31dfdfef1f1
I20250904 19:57:56.977279 11176 catalog_manager.cc:1554] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2: loading cluster ID for follower catalog manager: success
I20250904 19:57:56.984143 11449 raft_consensus.cc:357] T 9554457d24c044c78927500c90be27dc P 95b4a22c401f42148d5a6e4dde77a2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } }
I20250904 19:57:56.984848 11452 tablet_bootstrap.cc:492] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91: No bootstrap required, opened a new log
I20250904 19:57:56.984874 11449 raft_consensus.cc:383] T 9554457d24c044c78927500c90be27dc P 95b4a22c401f42148d5a6e4dde77a2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:56.985342 11449 raft_consensus.cc:738] T 9554457d24c044c78927500c90be27dc P 95b4a22c401f42148d5a6e4dde77a2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95b4a22c401f42148d5a6e4dde77a2f0, State: Initialized, Role: FOLLOWER
I20250904 19:57:56.985486 11452 ts_tablet_manager.cc:1397] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91: Time spent bootstrapping tablet: real 0.076s user 0.018s sys 0.037s
I20250904 19:57:56.988160 11452 raft_consensus.cc:357] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } }
I20250904 19:57:56.988840 11452 raft_consensus.cc:383] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:56.989133 11452 raft_consensus.cc:738] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3196a79eebf44a9b182b1b92b75fa91, State: Initialized, Role: FOLLOWER
I20250904 19:57:56.989866 11452 consensus_queue.cc:260] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } }
I20250904 19:57:56.986022 11449 consensus_queue.cc:260] T 9554457d24c044c78927500c90be27dc P 95b4a22c401f42148d5a6e4dde77a2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } }
I20250904 19:57:56.993712 11454 raft_consensus.cc:491] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:57:56.994196 11454 raft_consensus.cc:513] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } }
I20250904 19:57:56.995389 11452 ts_tablet_manager.cc:1428] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91: Time spent starting tablet: real 0.010s user 0.004s sys 0.003s
I20250904 19:57:56.996392 11452 tablet_bootstrap.cc:492] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91: Bootstrap starting.
I20250904 19:57:56.996557 11454 leader_election.cc:290] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59b040f0f42347a39a014d1a9a83eb22 (127.10.24.129:36707), a3196a79eebf44a9b182b1b92b75fa91 (127.10.24.130:45747)
I20250904 19:57:56.998950 11449 ts_tablet_manager.cc:1428] T 9554457d24c044c78927500c90be27dc P 95b4a22c401f42148d5a6e4dde77a2f0: Time spent starting tablet: real 0.054s user 0.006s sys 0.000s
I20250904 19:57:57.005239 11165 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e: acquiring CA information for follower catalog manager: success
I20250904 19:57:57.006615 11452 tablet_bootstrap.cc:654] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:56.989161 11176 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2: acquiring CA information for follower catalog manager: success
I20250904 19:57:57.014042 11176 catalog_manager.cc:1604] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:57:57.016124 11165 catalog_manager.cc:1604] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:57:57.019726 11449 tablet_bootstrap.cc:492] T 6da3727db70e4184a3f8751ae7b5fe17 P 95b4a22c401f42148d5a6e4dde77a2f0: Bootstrap starting.
I20250904 19:57:57.022116 11317 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "393ff9b87b4146f28b89225252e5277c" candidate_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3196a79eebf44a9b182b1b92b75fa91" is_pre_election: true
W20250904 19:57:57.023352 11358 leader_election.cc:343] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3196a79eebf44a9b182b1b92b75fa91 (127.10.24.130:45747): Illegal state: must be running to vote when last-logged opid is not known
I20250904 19:57:57.025537 11455 raft_consensus.cc:491] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:57:57.026029 11455 raft_consensus.cc:513] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } }
I20250904 19:57:57.027092 11239 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "393ff9b87b4146f28b89225252e5277c" candidate_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59b040f0f42347a39a014d1a9a83eb22" is_pre_election: true
I20250904 19:57:57.027814 11239 raft_consensus.cc:2466] T 393ff9b87b4146f28b89225252e5277c P 59b040f0f42347a39a014d1a9a83eb22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95b4a22c401f42148d5a6e4dde77a2f0 in term 0.
I20250904 19:57:57.028458 11455 leader_election.cc:290] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59b040f0f42347a39a014d1a9a83eb22 (127.10.24.129:36707), 95b4a22c401f42148d5a6e4dde77a2f0 (127.10.24.131:43909)
I20250904 19:57:57.028949 11358 leader_election.cc:304] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 59b040f0f42347a39a014d1a9a83eb22, 95b4a22c401f42148d5a6e4dde77a2f0; no voters: a3196a79eebf44a9b182b1b92b75fa91
I20250904 19:57:57.031917 11454 raft_consensus.cc:2802] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:57:57.032286 11454 raft_consensus.cc:491] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:57:57.032699 11454 raft_consensus.cc:3058] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:57.042608 11452 tablet_bootstrap.cc:492] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91: No bootstrap required, opened a new log
I20250904 19:57:57.032265 11449 tablet_bootstrap.cc:654] T 6da3727db70e4184a3f8751ae7b5fe17 P 95b4a22c401f42148d5a6e4dde77a2f0: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:57.045595 11452 ts_tablet_manager.cc:1397] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91: Time spent bootstrapping tablet: real 0.049s user 0.007s sys 0.017s
I20250904 19:57:57.049052 11452 raft_consensus.cc:357] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } }
I20250904 19:57:57.049058 11454 raft_consensus.cc:513] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } }
I20250904 19:57:57.049858 11452 raft_consensus.cc:383] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:57.050206 11452 raft_consensus.cc:738] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3196a79eebf44a9b182b1b92b75fa91, State: Initialized, Role: FOLLOWER
I20250904 19:57:57.050973 11452 consensus_queue.cc:260] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } }
I20250904 19:57:57.052073 11239 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "393ff9b87b4146f28b89225252e5277c" candidate_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59b040f0f42347a39a014d1a9a83eb22"
I20250904 19:57:57.052671 11239 raft_consensus.cc:3058] T 393ff9b87b4146f28b89225252e5277c P 59b040f0f42347a39a014d1a9a83eb22 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:57.055258 11452 ts_tablet_manager.cc:1428] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20250904 19:57:57.060019 11449 tablet_bootstrap.cc:492] T 6da3727db70e4184a3f8751ae7b5fe17 P 95b4a22c401f42148d5a6e4dde77a2f0: No bootstrap required, opened a new log
I20250904 19:57:57.062270 11454 leader_election.cc:290] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [CANDIDATE]: Term 1 election: Requested vote from peers 59b040f0f42347a39a014d1a9a83eb22 (127.10.24.129:36707), a3196a79eebf44a9b182b1b92b75fa91 (127.10.24.130:45747)
I20250904 19:57:57.063104 11449 ts_tablet_manager.cc:1397] T 6da3727db70e4184a3f8751ae7b5fe17 P 95b4a22c401f42148d5a6e4dde77a2f0: Time spent bootstrapping tablet: real 0.046s user 0.012s sys 0.000s
I20250904 19:57:57.065451 11449 raft_consensus.cc:357] T 6da3727db70e4184a3f8751ae7b5fe17 P 95b4a22c401f42148d5a6e4dde77a2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } }
I20250904 19:57:57.066133 11449 raft_consensus.cc:383] T 6da3727db70e4184a3f8751ae7b5fe17 P 95b4a22c401f42148d5a6e4dde77a2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:57.066426 11449 raft_consensus.cc:738] T 6da3727db70e4184a3f8751ae7b5fe17 P 95b4a22c401f42148d5a6e4dde77a2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95b4a22c401f42148d5a6e4dde77a2f0, State: Initialized, Role: FOLLOWER
I20250904 19:57:57.066628 11239 raft_consensus.cc:2466] T 393ff9b87b4146f28b89225252e5277c P 59b040f0f42347a39a014d1a9a83eb22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95b4a22c401f42148d5a6e4dde77a2f0 in term 1.
I20250904 19:57:57.068296 11449 consensus_queue.cc:260] T 6da3727db70e4184a3f8751ae7b5fe17 P 95b4a22c401f42148d5a6e4dde77a2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } }
I20250904 19:57:57.069139 11455 raft_consensus.cc:491] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:57:57.069578 11455 raft_consensus.cc:513] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } }
I20250904 19:57:57.070258 11358 leader_election.cc:304] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59b040f0f42347a39a014d1a9a83eb22, 95b4a22c401f42148d5a6e4dde77a2f0; no voters:
I20250904 19:57:57.070609 11449 ts_tablet_manager.cc:1428] T 6da3727db70e4184a3f8751ae7b5fe17 P 95b4a22c401f42148d5a6e4dde77a2f0: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20250904 19:57:57.070978 11454 raft_consensus.cc:2802] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:57:57.071465 11454 raft_consensus.cc:695] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [term 1 LEADER]: Becoming Leader. State: Replica: 95b4a22c401f42148d5a6e4dde77a2f0, State: Running, Role: LEADER
I20250904 19:57:57.072340 11454 consensus_queue.cc:237] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } }
I20250904 19:57:57.075670 11317 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "393ff9b87b4146f28b89225252e5277c" candidate_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3196a79eebf44a9b182b1b92b75fa91"
I20250904 19:57:57.076308 11317 raft_consensus.cc:3058] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:57.087414 11395 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6da3727db70e4184a3f8751ae7b5fe17" candidate_uuid: "a3196a79eebf44a9b182b1b92b75fa91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" is_pre_election: true
I20250904 19:57:57.087989 11395 raft_consensus.cc:2466] T 6da3727db70e4184a3f8751ae7b5fe17 P 95b4a22c401f42148d5a6e4dde77a2f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3196a79eebf44a9b182b1b92b75fa91 in term 0.
I20250904 19:57:57.089148 11395 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "9554457d24c044c78927500c90be27dc" candidate_uuid: "a3196a79eebf44a9b182b1b92b75fa91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" is_pre_election: true
I20250904 19:57:57.089716 11395 raft_consensus.cc:2466] T 9554457d24c044c78927500c90be27dc P 95b4a22c401f42148d5a6e4dde77a2f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3196a79eebf44a9b182b1b92b75fa91 in term 0.
I20250904 19:57:57.090581 11280 leader_election.cc:304] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95b4a22c401f42148d5a6e4dde77a2f0, a3196a79eebf44a9b182b1b92b75fa91; no voters:
I20250904 19:57:57.071403 11455 leader_election.cc:290] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 59b040f0f42347a39a014d1a9a83eb22 (127.10.24.129:36707), 95b4a22c401f42148d5a6e4dde77a2f0 (127.10.24.131:43909)
I20250904 19:57:57.091482 11455 raft_consensus.cc:2802] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:57:57.091892 11455 raft_consensus.cc:491] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:57:57.092059 11280 leader_election.cc:304] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95b4a22c401f42148d5a6e4dde77a2f0, a3196a79eebf44a9b182b1b92b75fa91; no voters:
I20250904 19:57:57.092252 11455 raft_consensus.cc:3058] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:57.092752 11468 raft_consensus.cc:2802] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:57:57.093183 11468 raft_consensus.cc:491] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:57:57.093557 11468 raft_consensus.cc:3058] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:57.099610 11317 raft_consensus.cc:2466] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95b4a22c401f42148d5a6e4dde77a2f0 in term 1.
I20250904 19:57:57.100226 11468 raft_consensus.cc:513] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } }
I20250904 19:57:57.102336 11468 leader_election.cc:290] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [CANDIDATE]: Term 1 election: Requested vote from peers 59b040f0f42347a39a014d1a9a83eb22 (127.10.24.129:36707), 95b4a22c401f42148d5a6e4dde77a2f0 (127.10.24.131:43909)
I20250904 19:57:57.103320 11395 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "9554457d24c044c78927500c90be27dc" candidate_uuid: "a3196a79eebf44a9b182b1b92b75fa91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95b4a22c401f42148d5a6e4dde77a2f0"
I20250904 19:57:57.100159 11466 raft_consensus.cc:491] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:57:57.103933 11395 raft_consensus.cc:3058] T 9554457d24c044c78927500c90be27dc P 95b4a22c401f42148d5a6e4dde77a2f0 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:57.103986 11466 raft_consensus.cc:513] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } }
I20250904 19:57:57.105196 11455 raft_consensus.cc:513] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } }
I20250904 19:57:57.119176 11395 raft_consensus.cc:2466] T 9554457d24c044c78927500c90be27dc P 95b4a22c401f42148d5a6e4dde77a2f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3196a79eebf44a9b182b1b92b75fa91 in term 1.
I20250904 19:57:57.120131 11455 leader_election.cc:290] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [CANDIDATE]: Term 1 election: Requested vote from peers 59b040f0f42347a39a014d1a9a83eb22 (127.10.24.129:36707), 95b4a22c401f42148d5a6e4dde77a2f0 (127.10.24.131:43909)
I20250904 19:57:57.121125 11466 leader_election.cc:290] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 59b040f0f42347a39a014d1a9a83eb22 (127.10.24.129:36707), 95b4a22c401f42148d5a6e4dde77a2f0 (127.10.24.131:43909)
I20250904 19:57:57.125314 11394 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "393ff9b87b4146f28b89225252e5277c" candidate_uuid: "a3196a79eebf44a9b182b1b92b75fa91" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" is_pre_election: true
I20250904 19:57:57.126551 11239 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6da3727db70e4184a3f8751ae7b5fe17" candidate_uuid: "a3196a79eebf44a9b182b1b92b75fa91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59b040f0f42347a39a014d1a9a83eb22" is_pre_election: true
I20250904 19:57:57.127027 11280 leader_election.cc:304] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95b4a22c401f42148d5a6e4dde77a2f0, a3196a79eebf44a9b182b1b92b75fa91; no voters:
I20250904 19:57:57.127869 11238 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "9554457d24c044c78927500c90be27dc" candidate_uuid: "a3196a79eebf44a9b182b1b92b75fa91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59b040f0f42347a39a014d1a9a83eb22" is_pre_election: true
I20250904 19:57:57.128552 11238 raft_consensus.cc:2466] T 9554457d24c044c78927500c90be27dc P 59b040f0f42347a39a014d1a9a83eb22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3196a79eebf44a9b182b1b92b75fa91 in term 0.
I20250904 19:57:57.129607 11237 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "9554457d24c044c78927500c90be27dc" candidate_uuid: "a3196a79eebf44a9b182b1b92b75fa91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59b040f0f42347a39a014d1a9a83eb22"
I20250904 19:57:57.130193 11237 raft_consensus.cc:3058] T 9554457d24c044c78927500c90be27dc P 59b040f0f42347a39a014d1a9a83eb22 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:57.131903 11236 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6da3727db70e4184a3f8751ae7b5fe17" candidate_uuid: "a3196a79eebf44a9b182b1b92b75fa91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59b040f0f42347a39a014d1a9a83eb22"
I20250904 19:57:57.133234 11235 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "393ff9b87b4146f28b89225252e5277c" candidate_uuid: "a3196a79eebf44a9b182b1b92b75fa91" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59b040f0f42347a39a014d1a9a83eb22" is_pre_election: true
I20250904 19:57:57.133888 11235 raft_consensus.cc:2466] T 393ff9b87b4146f28b89225252e5277c P 59b040f0f42347a39a014d1a9a83eb22 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3196a79eebf44a9b182b1b92b75fa91 in term 1.
I20250904 19:57:57.127183 11239 raft_consensus.cc:2466] T 6da3727db70e4184a3f8751ae7b5fe17 P 59b040f0f42347a39a014d1a9a83eb22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3196a79eebf44a9b182b1b92b75fa91 in term 0.
I20250904 19:57:57.136641 11280 leader_election.cc:304] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 59b040f0f42347a39a014d1a9a83eb22, a3196a79eebf44a9b182b1b92b75fa91; no voters: 95b4a22c401f42148d5a6e4dde77a2f0
I20250904 19:57:57.137281 11455 raft_consensus.cc:2802] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:57:57.137801 11455 raft_consensus.cc:695] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [term 1 LEADER]: Becoming Leader. State: Replica: a3196a79eebf44a9b182b1b92b75fa91, State: Running, Role: LEADER
I20250904 19:57:57.138145 11468 raft_consensus.cc:2802] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250904 19:57:57.138527 11468 raft_consensus.cc:491] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:57:57.138607 11455 consensus_queue.cc:237] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } }
I20250904 19:57:57.138927 11468 raft_consensus.cc:3058] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [term 1 FOLLOWER]: Advancing to term 2
I20250904 19:57:57.145505 11468 raft_consensus.cc:513] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } }
I20250904 19:57:57.149144 11468 leader_election.cc:290] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [CANDIDATE]: Term 2 election: Requested vote from peers 59b040f0f42347a39a014d1a9a83eb22 (127.10.24.129:36707), 95b4a22c401f42148d5a6e4dde77a2f0 (127.10.24.131:43909)
I20250904 19:57:57.150128 11394 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "393ff9b87b4146f28b89225252e5277c" candidate_uuid: "a3196a79eebf44a9b182b1b92b75fa91" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95b4a22c401f42148d5a6e4dde77a2f0"
I20250904 19:57:57.152237 11237 raft_consensus.cc:2466] T 9554457d24c044c78927500c90be27dc P 59b040f0f42347a39a014d1a9a83eb22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3196a79eebf44a9b182b1b92b75fa91 in term 1.
I20250904 19:57:57.152598 11239 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "393ff9b87b4146f28b89225252e5277c" candidate_uuid: "a3196a79eebf44a9b182b1b92b75fa91" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59b040f0f42347a39a014d1a9a83eb22"
I20250904 19:57:57.153213 11239 raft_consensus.cc:3058] T 393ff9b87b4146f28b89225252e5277c P 59b040f0f42347a39a014d1a9a83eb22 [term 1 FOLLOWER]: Advancing to term 2
I20250904 19:57:57.138983 10969 catalog_manager.cc:5582] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95b4a22c401f42148d5a6e4dde77a2f0 (127.10.24.131). New cstate: current_term: 1 leader_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } health_report { overall_health: UNKNOWN } } }
I20250904 19:57:57.158522 11395 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6da3727db70e4184a3f8751ae7b5fe17" candidate_uuid: "a3196a79eebf44a9b182b1b92b75fa91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95b4a22c401f42148d5a6e4dde77a2f0"
I20250904 19:57:57.159132 11395 raft_consensus.cc:3058] T 6da3727db70e4184a3f8751ae7b5fe17 P 95b4a22c401f42148d5a6e4dde77a2f0 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:57.162075 10971 catalog_manager.cc:5582] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3196a79eebf44a9b182b1b92b75fa91 (127.10.24.130). New cstate: current_term: 1 leader_uuid: "a3196a79eebf44a9b182b1b92b75fa91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } health_report { overall_health: UNKNOWN } } }
I20250904 19:57:57.164325 11239 raft_consensus.cc:2466] T 393ff9b87b4146f28b89225252e5277c P 59b040f0f42347a39a014d1a9a83eb22 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3196a79eebf44a9b182b1b92b75fa91 in term 2.
I20250904 19:57:57.166803 11280 leader_election.cc:304] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 59b040f0f42347a39a014d1a9a83eb22, a3196a79eebf44a9b182b1b92b75fa91; no voters: 95b4a22c401f42148d5a6e4dde77a2f0
I20250904 19:57:57.176406 11466 raft_consensus.cc:2802] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [term 2 FOLLOWER]: Leader election won for term 2
I20250904 19:57:57.176916 11466 raft_consensus.cc:695] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [term 2 LEADER]: Becoming Leader. State: Replica: a3196a79eebf44a9b182b1b92b75fa91, State: Running, Role: LEADER
I20250904 19:57:57.177685 11466 consensus_queue.cc:237] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } }
I20250904 19:57:57.184044 11395 raft_consensus.cc:2466] T 6da3727db70e4184a3f8751ae7b5fe17 P 95b4a22c401f42148d5a6e4dde77a2f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3196a79eebf44a9b182b1b92b75fa91 in term 1.
I20250904 19:57:57.185286 11280 leader_election.cc:304] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 95b4a22c401f42148d5a6e4dde77a2f0, a3196a79eebf44a9b182b1b92b75fa91; no voters: 59b040f0f42347a39a014d1a9a83eb22
I20250904 19:57:57.186142 11455 raft_consensus.cc:2802] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:57:57.186641 11455 raft_consensus.cc:695] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [term 1 LEADER]: Becoming Leader. State: Replica: a3196a79eebf44a9b182b1b92b75fa91, State: Running, Role: LEADER
I20250904 19:57:57.187458 11455 consensus_queue.cc:237] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } }
I20250904 19:57:57.231350 10971 catalog_manager.cc:5582] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 reported cstate change: term changed from 1 to 2, leader changed from 95b4a22c401f42148d5a6e4dde77a2f0 (127.10.24.131) to a3196a79eebf44a9b182b1b92b75fa91 (127.10.24.130). New cstate: current_term: 2 leader_uuid: "a3196a79eebf44a9b182b1b92b75fa91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } health_report { overall_health: HEALTHY } } }
I20250904 19:57:57.233261 10971 catalog_manager.cc:5582] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3196a79eebf44a9b182b1b92b75fa91 (127.10.24.130). New cstate: current_term: 1 leader_uuid: "a3196a79eebf44a9b182b1b92b75fa91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } health_report { overall_health: UNKNOWN } } }
I20250904 19:57:57.648363 11466 consensus_queue.cc:1035] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:57:57.659497 11239 raft_consensus.cc:1238] T 393ff9b87b4146f28b89225252e5277c P 59b040f0f42347a39a014d1a9a83eb22 [term 2 FOLLOWER]: Rejecting Update request from peer 95b4a22c401f42148d5a6e4dde77a2f0 for earlier term 1. Current term is 2. Ops: []
I20250904 19:57:57.665457 11491 consensus_queue.cc:1046] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:57:57.669447 11491 raft_consensus.cc:3053] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [term 1 LEADER]: Stepping down as leader of term 1
I20250904 19:57:57.669788 11491 raft_consensus.cc:738] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 95b4a22c401f42148d5a6e4dde77a2f0, State: Running, Role: LEADER
I20250904 19:57:57.670635 11491 consensus_queue.cc:260] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 } } peers { permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 } } peers { permanent_uuid: "a3196a79eebf44a9b182b1b92b75fa91" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45747 } }
I20250904 19:57:57.671662 11491 raft_consensus.cc:3058] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [term 1 FOLLOWER]: Advancing to term 2
I20250904 19:57:57.685252 11466 consensus_queue.cc:1035] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:57:57.699785 11466 consensus_queue.cc:1035] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:57:57.706887 11237 raft_consensus.cc:1273] T 393ff9b87b4146f28b89225252e5277c P 59b040f0f42347a39a014d1a9a83eb22 [term 2 FOLLOWER]: Refusing update from remote peer a3196a79eebf44a9b182b1b92b75fa91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20250904 19:57:57.710618 11466 consensus_queue.cc:1035] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:57:57.774132 11235 raft_consensus.cc:3058] T 6da3727db70e4184a3f8751ae7b5fe17 P 59b040f0f42347a39a014d1a9a83eb22 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:57.813005 11393 raft_consensus.cc:1273] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [term 2 FOLLOWER]: Refusing update from remote peer a3196a79eebf44a9b182b1b92b75fa91: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20250904 19:57:57.813477 11393 pending_rounds.cc:85] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0: Aborting all ops after (but not including) 0
I20250904 19:57:57.813766 11393 pending_rounds.cc:107] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20250904 19:57:57.814054 11393 raft_consensus.cc:2887] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20250904 19:57:57.821429 11496 consensus_queue.cc:1035] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "59b040f0f42347a39a014d1a9a83eb22" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250904 19:57:57.847810 11466 consensus_queue.cc:1035] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "95b4a22c401f42148d5a6e4dde77a2f0" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 43909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:57:58.090240 11227 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250904 19:57:58.102994 11384 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250904 19:57:58.110891 11307 tablet_service.cc:1430] Tablet server has 3 leaders and 0 scanners
I20250904 19:57:58.451984 10338 tablet_server.cc:178] TabletServer@127.10.24.129:0 shutting down...
I20250904 19:57:58.495381 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:57:58.495916 10338 tablet_replica.cc:331] T 9554457d24c044c78927500c90be27dc P 59b040f0f42347a39a014d1a9a83eb22: stopping tablet replica
I20250904 19:57:58.496479 10338 raft_consensus.cc:2241] T 9554457d24c044c78927500c90be27dc P 59b040f0f42347a39a014d1a9a83eb22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:57:58.496953 10338 raft_consensus.cc:2270] T 9554457d24c044c78927500c90be27dc P 59b040f0f42347a39a014d1a9a83eb22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:58.500759 10338 tablet_replica.cc:331] T 393ff9b87b4146f28b89225252e5277c P 59b040f0f42347a39a014d1a9a83eb22: stopping tablet replica
I20250904 19:57:58.501336 10338 raft_consensus.cc:2241] T 393ff9b87b4146f28b89225252e5277c P 59b040f0f42347a39a014d1a9a83eb22 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250904 19:57:58.501834 10338 raft_consensus.cc:2270] T 393ff9b87b4146f28b89225252e5277c P 59b040f0f42347a39a014d1a9a83eb22 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:58.506043 10338 tablet_replica.cc:331] T 6da3727db70e4184a3f8751ae7b5fe17 P 59b040f0f42347a39a014d1a9a83eb22: stopping tablet replica
I20250904 19:57:58.506675 10338 raft_consensus.cc:2241] T 6da3727db70e4184a3f8751ae7b5fe17 P 59b040f0f42347a39a014d1a9a83eb22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:57:58.507148 10338 raft_consensus.cc:2270] T 6da3727db70e4184a3f8751ae7b5fe17 P 59b040f0f42347a39a014d1a9a83eb22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:58.529862 10338 tablet_server.cc:195] TabletServer@127.10.24.129:0 shutdown complete.
I20250904 19:57:58.543967 10338 tablet_server.cc:178] TabletServer@127.10.24.130:0 shutting down...
I20250904 19:57:58.571260 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:57:58.571871 10338 tablet_replica.cc:331] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91: stopping tablet replica
I20250904 19:57:58.572585 10338 raft_consensus.cc:2241] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [term 2 LEADER]: Raft consensus shutting down.
I20250904 19:57:58.573239 10338 raft_consensus.cc:2270] T 393ff9b87b4146f28b89225252e5277c P a3196a79eebf44a9b182b1b92b75fa91 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:58.575089 10338 tablet_replica.cc:331] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91: stopping tablet replica
I20250904 19:57:58.575529 10338 raft_consensus.cc:2241] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:57:58.576179 10338 raft_consensus.cc:2270] T 6da3727db70e4184a3f8751ae7b5fe17 P a3196a79eebf44a9b182b1b92b75fa91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:58.578356 10338 tablet_replica.cc:331] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91: stopping tablet replica
I20250904 19:57:58.578802 10338 raft_consensus.cc:2241] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:57:58.579541 10338 raft_consensus.cc:2270] T 9554457d24c044c78927500c90be27dc P a3196a79eebf44a9b182b1b92b75fa91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:58.599436 10338 tablet_server.cc:195] TabletServer@127.10.24.130:0 shutdown complete.
I20250904 19:57:58.608816 10338 tablet_server.cc:178] TabletServer@127.10.24.131:0 shutting down...
I20250904 19:57:58.626972 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:57:58.627539 10338 tablet_replica.cc:331] T 6da3727db70e4184a3f8751ae7b5fe17 P 95b4a22c401f42148d5a6e4dde77a2f0: stopping tablet replica
I20250904 19:57:58.628173 10338 raft_consensus.cc:2241] T 6da3727db70e4184a3f8751ae7b5fe17 P 95b4a22c401f42148d5a6e4dde77a2f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:57:58.628551 10338 raft_consensus.cc:2270] T 6da3727db70e4184a3f8751ae7b5fe17 P 95b4a22c401f42148d5a6e4dde77a2f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:58.630324 10338 tablet_replica.cc:331] T 9554457d24c044c78927500c90be27dc P 95b4a22c401f42148d5a6e4dde77a2f0: stopping tablet replica
I20250904 19:57:58.630843 10338 raft_consensus.cc:2241] T 9554457d24c044c78927500c90be27dc P 95b4a22c401f42148d5a6e4dde77a2f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:57:58.631206 10338 raft_consensus.cc:2270] T 9554457d24c044c78927500c90be27dc P 95b4a22c401f42148d5a6e4dde77a2f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:58.633051 10338 tablet_replica.cc:331] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0: stopping tablet replica
I20250904 19:57:58.633473 10338 raft_consensus.cc:2241] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250904 19:57:58.633821 10338 raft_consensus.cc:2270] T 393ff9b87b4146f28b89225252e5277c P 95b4a22c401f42148d5a6e4dde77a2f0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:58.652876 10338 tablet_server.cc:195] TabletServer@127.10.24.131:0 shutdown complete.
I20250904 19:57:58.663916 10338 master.cc:561] Master@127.10.24.190:36775 shutting down...
I20250904 19:57:58.678659 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:57:58.679425 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:58.679780 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 8092d6d8791340cf9e5e9c08d962e0c2: stopping tablet replica
I20250904 19:57:58.697728 10338 master.cc:583] Master@127.10.24.190:36775 shutdown complete.
I20250904 19:57:58.708762 10338 master.cc:561] Master@127.10.24.189:39855 shutting down...
I20250904 19:57:58.721400 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:57:58.721930 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:58.722218 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 1fef062f07b7422098d5a5d189809b6e: stopping tablet replica
I20250904 19:57:58.740355 10338 master.cc:583] Master@127.10.24.189:39855 shutdown complete.
I20250904 19:57:58.749501 10338 master.cc:561] Master@127.10.24.188:41619 shutting down...
I20250904 19:57:58.763629 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:57:58.764110 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:57:58.764389 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 4179ea0edfa5437ba65276fe8add60f2: stopping tablet replica
I20250904 19:57:58.781711 10338 master.cc:583] Master@127.10.24.188:41619 shutdown complete.
[ OK ] RemoteKsckTest.TestCheckUnusualFlags (3175 ms)
[ RUN ] RemoteKsckTest.TestTabletServerMismatchedUUID
I20250904 19:57:58.806922 10338 test_util.cc:276] Using random seed: 592588150
I20250904 19:57:58.807829 10338 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.24.190:45327,127.10.24.189:40465,127.10.24.188:41645
I20250904 19:57:58.808996 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:57:58.813340 11535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:57:58.813465 11536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:57:58.814553 11538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:58.815892 10338 server_base.cc:1047] running on GCE node
I20250904 19:57:58.816691 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:57:58.816869 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:57:58.817003 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015878816987 us; error 0 us; skew 500 ppm
I20250904 19:57:58.817456 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:57:58.819398 10338 webserver.cc:480] Webserver started at http://127.10.24.190:34785/ using document root <none> and password file <none>
I20250904 19:57:58.819849 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:57:58.820010 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:57:58.820242 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:57:58.821161 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/master-0-root/instance:
uuid: "5e7da9c2711646749a203ea10baf03a9"
format_stamp: "Formatted at 2025-09-04 19:57:58 on dist-test-slave-f3xj"
I20250904 19:57:58.825191 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250904 19:57:58.828044 11543 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:58.828665 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250904 19:57:58.828909 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/master-0-root
uuid: "5e7da9c2711646749a203ea10baf03a9"
format_stamp: "Formatted at 2025-09-04 19:57:58 on dist-test-slave-f3xj"
I20250904 19:57:58.829161 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:57:58.840425 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:57:58.841302 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:57:58.872828 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.190:45327
I20250904 19:57:58.872920 11594 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.190:45327 every 8 connection(s)
I20250904 19:57:58.876282 11595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:58.876425 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:57:58.881234 11597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:57:58.882048 11598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:58.883091 11595 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:57:58.883630 11600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:58.886689 10338 server_base.cc:1047] running on GCE node
I20250904 19:57:58.887537 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:57:58.887749 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:57:58.887902 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015878887889 us; error 0 us; skew 500 ppm
I20250904 19:57:58.888383 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:57:58.890604 10338 webserver.cc:480] Webserver started at http://127.10.24.189:33601/ using document root <none> and password file <none>
I20250904 19:57:58.891042 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:57:58.891211 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:57:58.891441 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:57:58.892505 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/master-1-root/instance:
uuid: "c0ee841362264b408de443e2f2eae8f3"
format_stamp: "Formatted at 2025-09-04 19:57:58 on dist-test-slave-f3xj"
I20250904 19:57:58.894757 11595 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:57:58.896917 11544 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.24.189:40465: connect: Connection refused (error 111) [suppressed 12 similar messages]
I20250904 19:57:58.897243 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
W20250904 19:57:58.899593 11595 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.189:40465: Network error: Client connection negotiation failed: client connection to 127.10.24.189:40465: connect: Connection refused (error 111)
I20250904 19:57:58.900979 11608 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:58.901618 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20250904 19:57:58.901872 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/master-1-root
uuid: "c0ee841362264b408de443e2f2eae8f3"
format_stamp: "Formatted at 2025-09-04 19:57:58 on dist-test-slave-f3xj"
I20250904 19:57:58.902107 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:57:58.909507 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:57:58.910351 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:57:58.918030 11595 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } attempt: 1
W20250904 19:57:58.921840 11595 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.189:40465: Network error: Client connection negotiation failed: client connection to 127.10.24.189:40465: connect: Connection refused (error 111)
I20250904 19:57:58.943349 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.189:40465
I20250904 19:57:58.943446 11659 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.189:40465 every 8 connection(s)
I20250904 19:57:58.946976 11660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:58.947064 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:57:58.952306 11662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:57:58.953097 11663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:58.954293 11660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:57:58.954764 11665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:58.955699 10338 server_base.cc:1047] running on GCE node
I20250904 19:57:58.956676 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:57:58.956926 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:57:58.957080 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015878957064 us; error 0 us; skew 500 ppm
I20250904 19:57:58.957577 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:57:58.960068 10338 webserver.cc:480] Webserver started at http://127.10.24.188:33007/ using document root <none> and password file <none>
I20250904 19:57:58.960580 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:57:58.960801 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:57:58.961099 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:57:58.962384 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/master-2-root/instance:
uuid: "082185d1cb27408b8bf75c9b69857cfa"
format_stamp: "Formatted at 2025-09-04 19:57:58 on dist-test-slave-f3xj"
I20250904 19:57:58.965411 11660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:57:58.967530 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20250904 19:57:58.971079 11672 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:58.971853 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250904 19:57:58.972106 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/master-2-root
uuid: "082185d1cb27408b8bf75c9b69857cfa"
format_stamp: "Formatted at 2025-09-04 19:57:58 on dist-test-slave-f3xj"
I20250904 19:57:58.972368 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:57:58.976137 11660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:57:58.980888 11660 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:41645: Network error: Client connection negotiation failed: client connection to 127.10.24.188:41645: connect: Connection refused (error 111)
I20250904 19:57:58.987319 11595 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } attempt: 2
I20250904 19:57:58.996407 11595 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:57:59.000396 11595 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:41645: Network error: Client connection negotiation failed: client connection to 127.10.24.188:41645: connect: Connection refused (error 111)
I20250904 19:57:59.001636 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:57:59.002635 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:57:59.019316 11660 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } attempt: 1
W20250904 19:57:59.023857 11660 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:41645: Network error: Client connection negotiation failed: client connection to 127.10.24.188:41645: connect: Connection refused (error 111)
I20250904 19:57:59.024832 11595 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } attempt: 1
W20250904 19:57:59.029016 11595 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:41645: Network error: Client connection negotiation failed: client connection to 127.10.24.188:41645: connect: Connection refused (error 111)
I20250904 19:57:59.037957 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.188:41645
I20250904 19:57:59.038053 11724 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.188:41645 every 8 connection(s)
I20250904 19:57:59.040771 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 19:57:59.041450 11725 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:59.046855 11725 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:57:59.056377 11725 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:57:59.064814 11725 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:57:59.067369 11660 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } attempt: 2
I20250904 19:57:59.080132 11725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa: Bootstrap starting.
I20250904 19:57:59.080996 11660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3: Bootstrap starting.
I20250904 19:57:59.084234 11725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:59.084945 11660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:59.088255 11725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa: No bootstrap required, opened a new log
I20250904 19:57:59.088704 11660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3: No bootstrap required, opened a new log
I20250904 19:57:59.090117 11725 raft_consensus.cc:357] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e7da9c2711646749a203ea10baf03a9" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } } peers { permanent_uuid: "c0ee841362264b408de443e2f2eae8f3" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } } peers { permanent_uuid: "082185d1cb27408b8bf75c9b69857cfa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } }
I20250904 19:57:59.090633 11725 raft_consensus.cc:383] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:59.090488 11660 raft_consensus.cc:357] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e7da9c2711646749a203ea10baf03a9" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } } peers { permanent_uuid: "c0ee841362264b408de443e2f2eae8f3" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } } peers { permanent_uuid: "082185d1cb27408b8bf75c9b69857cfa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } }
I20250904 19:57:59.090916 11725 raft_consensus.cc:738] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 082185d1cb27408b8bf75c9b69857cfa, State: Initialized, Role: FOLLOWER
I20250904 19:57:59.091095 11660 raft_consensus.cc:383] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:59.091382 11660 raft_consensus.cc:738] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0ee841362264b408de443e2f2eae8f3, State: Initialized, Role: FOLLOWER
I20250904 19:57:59.091542 11725 consensus_queue.cc:260] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e7da9c2711646749a203ea10baf03a9" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } } peers { permanent_uuid: "c0ee841362264b408de443e2f2eae8f3" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } } peers { permanent_uuid: "082185d1cb27408b8bf75c9b69857cfa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } }
I20250904 19:57:59.091992 11660 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e7da9c2711646749a203ea10baf03a9" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } } peers { permanent_uuid: "c0ee841362264b408de443e2f2eae8f3" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } } peers { permanent_uuid: "082185d1cb27408b8bf75c9b69857cfa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } }
I20250904 19:57:59.093447 11732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e7da9c2711646749a203ea10baf03a9" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } } peers { permanent_uuid: "c0ee841362264b408de443e2f2eae8f3" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } } peers { permanent_uuid: "082185d1cb27408b8bf75c9b69857cfa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } } }
I20250904 19:57:59.094247 11732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:59.093901 11733 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e7da9c2711646749a203ea10baf03a9" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } } peers { permanent_uuid: "c0ee841362264b408de443e2f2eae8f3" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } } peers { permanent_uuid: "082185d1cb27408b8bf75c9b69857cfa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } } }
I20250904 19:57:59.094751 11733 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:59.095011 11725 sys_catalog.cc:564] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:57:59.095187 11660 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:57:59.095440 11595 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } attempt: 2
I20250904 19:57:59.115319 11595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9: Bootstrap starting.
W20250904 19:57:59.119405 11750 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:57:59.119814 11750 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250904 19:57:59.121567 11595 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9: Neither blocks nor log segments found. Creating new log.
W20250904 19:57:59.125456 11755 catalog_manager.cc:1560] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20250904 19:57:59.125617 11595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9: No bootstrap required, opened a new log
W20250904 19:57:59.125732 11755 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250904 19:57:59.127683 11595 raft_consensus.cc:357] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e7da9c2711646749a203ea10baf03a9" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } } peers { permanent_uuid: "c0ee841362264b408de443e2f2eae8f3" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } } peers { permanent_uuid: "082185d1cb27408b8bf75c9b69857cfa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } }
I20250904 19:57:59.128194 11595 raft_consensus.cc:383] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:57:59.128401 11595 raft_consensus.cc:738] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e7da9c2711646749a203ea10baf03a9, State: Initialized, Role: FOLLOWER
I20250904 19:57:59.128906 11595 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e7da9c2711646749a203ea10baf03a9" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } } peers { permanent_uuid: "c0ee841362264b408de443e2f2eae8f3" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } } peers { permanent_uuid: "082185d1cb27408b8bf75c9b69857cfa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } }
I20250904 19:57:59.130381 11757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e7da9c2711646749a203ea10baf03a9" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } } peers { permanent_uuid: "c0ee841362264b408de443e2f2eae8f3" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } } peers { permanent_uuid: "082185d1cb27408b8bf75c9b69857cfa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } } }
I20250904 19:57:59.130903 11757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:59.131541 11595 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:57:59.140604 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20250904 19:57:59.140800 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20250904 19:57:59.142025 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:57:59.143285 11768 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:57:59.143553 11768 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20250904 19:57:59.147390 11769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:57:59.148500 11770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:59.149660 10338 server_base.cc:1047] running on GCE node
W20250904 19:57:59.150094 11772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:59.150810 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:57:59.151011 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:57:59.151172 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015879151155 us; error 0 us; skew 500 ppm
I20250904 19:57:59.151742 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:57:59.153759 10338 webserver.cc:480] Webserver started at http://127.10.24.129:42809/ using document root <none> and password file <none>
I20250904 19:57:59.154220 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:57:59.154390 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:57:59.154618 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:57:59.155550 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/ts-0-root/instance:
uuid: "adf3a8303f054fdd83dca3a2b60f0684"
format_stamp: "Formatted at 2025-09-04 19:57:59 on dist-test-slave-f3xj"
I20250904 19:57:59.159507 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20250904 19:57:59.162369 11777 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:59.163034 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250904 19:57:59.163295 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/ts-0-root
uuid: "adf3a8303f054fdd83dca3a2b60f0684"
format_stamp: "Formatted at 2025-09-04 19:57:59 on dist-test-slave-f3xj"
I20250904 19:57:59.163542 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:57:59.178086 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:57:59.179047 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:57:59.180349 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:57:59.182394 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:57:59.182583 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:59.182791 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:57:59.182945 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:59.217459 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.129:41455
I20250904 19:57:59.217542 11839 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.129:41455 every 8 connection(s)
I20250904 19:57:59.230443 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:57:59.234829 11840 heartbeater.cc:344] Connected to a master server at 127.10.24.188:41645
I20250904 19:57:59.235247 11840 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:59.236063 11840 heartbeater.cc:507] Master 127.10.24.188:41645 requested a full tablet report, sending...
I20250904 19:57:59.239053 11690 ts_manager.cc:194] Registered new tserver with Master: adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129:41455)
I20250904 19:57:59.239511 11841 heartbeater.cc:344] Connected to a master server at 127.10.24.189:40465
I20250904 19:57:59.239948 11841 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:59.240850 11841 heartbeater.cc:507] Master 127.10.24.189:40465 requested a full tablet report, sending...
W20250904 19:57:59.240967 11847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:57:59.242681 11848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:59.243839 11842 heartbeater.cc:344] Connected to a master server at 127.10.24.190:45327
I20250904 19:57:59.244251 11842 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:59.245074 11842 heartbeater.cc:507] Master 127.10.24.190:45327 requested a full tablet report, sending...
I20250904 19:57:59.245610 11625 ts_manager.cc:194] Registered new tserver with Master: adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129:41455)
I20250904 19:57:59.247777 11560 ts_manager.cc:194] Registered new tserver with Master: adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129:41455)
W20250904 19:57:59.248953 11850 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:59.249820 10338 server_base.cc:1047] running on GCE node
I20250904 19:57:59.250468 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:57:59.250653 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:57:59.250813 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015879250797 us; error 0 us; skew 500 ppm
I20250904 19:57:59.251412 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:57:59.253721 10338 webserver.cc:480] Webserver started at http://127.10.24.130:38483/ using document root <none> and password file <none>
I20250904 19:57:59.254163 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:57:59.254318 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:57:59.254552 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:57:59.255492 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/ts-1-root/instance:
uuid: "f57a6e99126a4394821e0f0c1e2eb8aa"
format_stamp: "Formatted at 2025-09-04 19:57:59 on dist-test-slave-f3xj"
I20250904 19:57:59.258687 11757 raft_consensus.cc:491] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:57:59.259140 11757 raft_consensus.cc:513] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e7da9c2711646749a203ea10baf03a9" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } } peers { permanent_uuid: "c0ee841362264b408de443e2f2eae8f3" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } } peers { permanent_uuid: "082185d1cb27408b8bf75c9b69857cfa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } }
I20250904 19:57:59.259639 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20250904 19:57:59.261076 11757 leader_election.cc:290] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0ee841362264b408de443e2f2eae8f3 (127.10.24.189:40465), 082185d1cb27408b8bf75c9b69857cfa (127.10.24.188:41645)
I20250904 19:57:59.262676 11635 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e7da9c2711646749a203ea10baf03a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0ee841362264b408de443e2f2eae8f3" is_pre_election: true
I20250904 19:57:59.262779 11700 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e7da9c2711646749a203ea10baf03a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "082185d1cb27408b8bf75c9b69857cfa" is_pre_election: true
I20250904 19:57:59.263316 11635 raft_consensus.cc:2466] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e7da9c2711646749a203ea10baf03a9 in term 0.
I20250904 19:57:59.263464 11700 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e7da9c2711646749a203ea10baf03a9 in term 0.
I20250904 19:57:59.264534 11544 leader_election.cc:304] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e7da9c2711646749a203ea10baf03a9, c0ee841362264b408de443e2f2eae8f3; no voters:
I20250904 19:57:59.265250 11757 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:57:59.265717 11757 raft_consensus.cc:491] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:57:59.266108 11757 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:59.266296 11855 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:59.267095 10338 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.003s sys 0.003s
I20250904 19:57:59.267357 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/ts-1-root
uuid: "f57a6e99126a4394821e0f0c1e2eb8aa"
format_stamp: "Formatted at 2025-09-04 19:57:59 on dist-test-slave-f3xj"
I20250904 19:57:59.267612 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:57:59.270800 11757 raft_consensus.cc:513] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e7da9c2711646749a203ea10baf03a9" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } } peers { permanent_uuid: "c0ee841362264b408de443e2f2eae8f3" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } } peers { permanent_uuid: "082185d1cb27408b8bf75c9b69857cfa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } }
I20250904 19:57:59.272634 11757 leader_election.cc:290] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [CANDIDATE]: Term 1 election: Requested vote from peers c0ee841362264b408de443e2f2eae8f3 (127.10.24.189:40465), 082185d1cb27408b8bf75c9b69857cfa (127.10.24.188:41645)
I20250904 19:57:59.273548 11635 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e7da9c2711646749a203ea10baf03a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0ee841362264b408de443e2f2eae8f3"
I20250904 19:57:59.274111 11635 raft_consensus.cc:3058] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:59.274600 11700 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5e7da9c2711646749a203ea10baf03a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "082185d1cb27408b8bf75c9b69857cfa"
I20250904 19:57:59.275105 11700 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:57:59.279628 11635 raft_consensus.cc:2466] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e7da9c2711646749a203ea10baf03a9 in term 1.
I20250904 19:57:59.280652 11544 leader_election.cc:304] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e7da9c2711646749a203ea10baf03a9, c0ee841362264b408de443e2f2eae8f3; no voters:
I20250904 19:57:59.281677 11757 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:57:59.282363 11700 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e7da9c2711646749a203ea10baf03a9 in term 1.
I20250904 19:57:59.282842 11757 raft_consensus.cc:695] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [term 1 LEADER]: Becoming Leader. State: Replica: 5e7da9c2711646749a203ea10baf03a9, State: Running, Role: LEADER
I20250904 19:57:59.283618 11757 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e7da9c2711646749a203ea10baf03a9" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } } peers { permanent_uuid: "c0ee841362264b408de443e2f2eae8f3" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } } peers { permanent_uuid: "082185d1cb27408b8bf75c9b69857cfa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } }
I20250904 19:57:59.287827 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:57:59.288779 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:57:59.288509 11860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e7da9c2711646749a203ea10baf03a9. Latest consensus state: current_term: 1 leader_uuid: "5e7da9c2711646749a203ea10baf03a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e7da9c2711646749a203ea10baf03a9" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } } peers { permanent_uuid: "c0ee841362264b408de443e2f2eae8f3" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } } peers { permanent_uuid: "082185d1cb27408b8bf75c9b69857cfa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } } }
I20250904 19:57:59.289113 11860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [sys.catalog]: This master's current role is: LEADER
I20250904 19:57:59.290675 11862 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 19:57:59.290871 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:57:59.293922 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:57:59.294171 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:59.294440 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:57:59.294641 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:59.297044 11862 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 19:57:59.307451 11700 raft_consensus.cc:1273] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa [term 1 FOLLOWER]: Refusing update from remote peer 5e7da9c2711646749a203ea10baf03a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:57:59.308259 11635 raft_consensus.cc:1273] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3 [term 1 FOLLOWER]: Refusing update from remote peer 5e7da9c2711646749a203ea10baf03a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:57:59.309127 11860 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "082185d1cb27408b8bf75c9b69857cfa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:57:59.311976 11860 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0ee841362264b408de443e2f2eae8f3" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:57:59.330121 11733 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e7da9c2711646749a203ea10baf03a9. Latest consensus state: current_term: 1 leader_uuid: "5e7da9c2711646749a203ea10baf03a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e7da9c2711646749a203ea10baf03a9" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } } peers { permanent_uuid: "c0ee841362264b408de443e2f2eae8f3" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } } peers { permanent_uuid: "082185d1cb27408b8bf75c9b69857cfa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } } }
I20250904 19:57:59.330914 11733 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:59.341370 11733 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5e7da9c2711646749a203ea10baf03a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e7da9c2711646749a203ea10baf03a9" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } } peers { permanent_uuid: "c0ee841362264b408de443e2f2eae8f3" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } } peers { permanent_uuid: "082185d1cb27408b8bf75c9b69857cfa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } } }
I20250904 19:57:59.342067 11733 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:59.343593 11732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e7da9c2711646749a203ea10baf03a9. Latest consensus state: current_term: 1 leader_uuid: "5e7da9c2711646749a203ea10baf03a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e7da9c2711646749a203ea10baf03a9" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } } peers { permanent_uuid: "c0ee841362264b408de443e2f2eae8f3" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } } peers { permanent_uuid: "082185d1cb27408b8bf75c9b69857cfa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } } }
I20250904 19:57:59.344321 11732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:59.350980 11862 catalog_manager.cc:1349] Generated new cluster ID: 96c9aabb12c346dfae562abcf50e9800
I20250904 19:57:59.351341 11862 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 19:57:59.352555 11757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5e7da9c2711646749a203ea10baf03a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e7da9c2711646749a203ea10baf03a9" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } } peers { permanent_uuid: "c0ee841362264b408de443e2f2eae8f3" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } } peers { permanent_uuid: "082185d1cb27408b8bf75c9b69857cfa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } } }
I20250904 19:57:59.353403 11757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [sys.catalog]: This master's current role is: LEADER
I20250904 19:57:59.354058 11860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5e7da9c2711646749a203ea10baf03a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e7da9c2711646749a203ea10baf03a9" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } } peers { permanent_uuid: "c0ee841362264b408de443e2f2eae8f3" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } } peers { permanent_uuid: "082185d1cb27408b8bf75c9b69857cfa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } } }
I20250904 19:57:59.354707 11860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [sys.catalog]: This master's current role is: LEADER
I20250904 19:57:59.360647 11732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5e7da9c2711646749a203ea10baf03a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e7da9c2711646749a203ea10baf03a9" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 45327 } } peers { permanent_uuid: "c0ee841362264b408de443e2f2eae8f3" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40465 } } peers { permanent_uuid: "082185d1cb27408b8bf75c9b69857cfa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41645 } } }
I20250904 19:57:59.361337 11732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:57:59.381081 11862 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 19:57:59.382822 11862 catalog_manager.cc:1506] Loading token signing keys...
I20250904 19:57:59.414077 11862 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9: Generated new TSK 0
I20250904 19:57:59.414798 11862 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 19:57:59.420727 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.130:44159
I20250904 19:57:59.420874 11929 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.130:44159 every 8 connection(s)
I20250904 19:57:59.427821 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:57:59.433739 11560 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33532
I20250904 19:57:59.445992 11932 heartbeater.cc:344] Connected to a master server at 127.10.24.190:45327
I20250904 19:57:59.446429 11932 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:59.447275 11932 heartbeater.cc:507] Master 127.10.24.190:45327 requested a full tablet report, sending...
I20250904 19:57:59.447865 11842 heartbeater.cc:499] Master 127.10.24.190:45327 was elected leader, sending a full tablet report...
I20250904 19:57:59.453279 11560 ts_manager.cc:194] Registered new tserver with Master: f57a6e99126a4394821e0f0c1e2eb8aa (127.10.24.130:44159)
I20250904 19:57:59.454856 11560 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33534
I20250904 19:57:59.459530 11930 heartbeater.cc:344] Connected to a master server at 127.10.24.188:41645
I20250904 19:57:59.459947 11930 heartbeater.cc:461] Registering TS with master...
W20250904 19:57:59.460414 11937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:59.460593 11930 heartbeater.cc:507] Master 127.10.24.188:41645 requested a full tablet report, sending...
I20250904 19:57:59.463701 11690 ts_manager.cc:194] Registered new tserver with Master: f57a6e99126a4394821e0f0c1e2eb8aa (127.10.24.130:44159)
I20250904 19:57:59.467545 11932 heartbeater.cc:499] Master 127.10.24.190:45327 was elected leader, sending a full tablet report...
I20250904 19:57:59.470412 11931 heartbeater.cc:344] Connected to a master server at 127.10.24.189:40465
I20250904 19:57:59.470789 11931 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:59.472375 11931 heartbeater.cc:507] Master 127.10.24.189:40465 requested a full tablet report, sending...
W20250904 19:57:59.476600 11938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:59.477516 11625 ts_manager.cc:194] Registered new tserver with Master: f57a6e99126a4394821e0f0c1e2eb8aa (127.10.24.130:44159)
W20250904 19:57:59.478945 11940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:57:59.480047 10338 server_base.cc:1047] running on GCE node
I20250904 19:57:59.481019 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:57:59.481292 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:57:59.481509 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015879481490 us; error 0 us; skew 500 ppm
I20250904 19:57:59.482196 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:57:59.484427 10338 webserver.cc:480] Webserver started at http://127.10.24.131:37541/ using document root <none> and password file <none>
I20250904 19:57:59.484872 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:57:59.485051 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:57:59.485373 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:57:59.486568 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/ts-2-root/instance:
uuid: "56bd53feb0344b60852adb4017f906bc"
format_stamp: "Formatted at 2025-09-04 19:57:59 on dist-test-slave-f3xj"
I20250904 19:57:59.491451 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20250904 19:57:59.495149 11945 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:59.495864 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250904 19:57:59.496117 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/ts-2-root
uuid: "56bd53feb0344b60852adb4017f906bc"
format_stamp: "Formatted at 2025-09-04 19:57:59 on dist-test-slave-f3xj"
I20250904 19:57:59.496340 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:57:59.517645 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:57:59.518970 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:57:59.520761 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:57:59.522949 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:57:59.523310 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:59.523710 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:57:59.524014 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:57:59.580546 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.131:37237
I20250904 19:57:59.580649 12007 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.131:37237 every 8 connection(s)
I20250904 19:57:59.627696 12008 heartbeater.cc:344] Connected to a master server at 127.10.24.188:41645
I20250904 19:57:59.628070 12008 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:59.628839 12008 heartbeater.cc:507] Master 127.10.24.188:41645 requested a full tablet report, sending...
I20250904 19:57:59.632143 11689 ts_manager.cc:194] Registered new tserver with Master: 56bd53feb0344b60852adb4017f906bc (127.10.24.131:37237)
I20250904 19:57:59.634658 12010 heartbeater.cc:344] Connected to a master server at 127.10.24.190:45327
I20250904 19:57:59.635068 12010 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:59.635943 12010 heartbeater.cc:507] Master 127.10.24.190:45327 requested a full tablet report, sending...
I20250904 19:57:59.687718 11558 ts_manager.cc:194] Registered new tserver with Master: 56bd53feb0344b60852adb4017f906bc (127.10.24.131:37237)
I20250904 19:57:59.690711 12009 heartbeater.cc:344] Connected to a master server at 127.10.24.189:40465
I20250904 19:57:59.691344 12009 heartbeater.cc:461] Registering TS with master...
I20250904 19:57:59.690821 11558 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33540
I20250904 19:57:59.692816 12009 heartbeater.cc:507] Master 127.10.24.189:40465 requested a full tablet report, sending...
I20250904 19:57:59.695706 11624 ts_manager.cc:194] Registered new tserver with Master: 56bd53feb0344b60852adb4017f906bc (127.10.24.131:37237)
I20250904 19:57:59.696504 10338 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.104566626s
I20250904 19:57:59.705549 12010 heartbeater.cc:499] Master 127.10.24.190:45327 was elected leader, sending a full tablet report...
I20250904 19:57:59.756495 11558 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33550:
name: "ksck-test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\n\000\000\000\004\001\024\000\000\000""\004\001\n\000\000\000\004\001\024\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 19:57:59.760496 11558 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table ksck-test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 19:57:59.864554 11973 tablet_service.cc:1468] Processing CreateTablet for tablet 9bae8bd3195540a4b394e0b851afdb67 (DEFAULT_TABLE table=ksck-test-table [id=9823dbf15cb24b85a2b2ba6e85893d4c]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:57:59.866348 11973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bae8bd3195540a4b394e0b851afdb67. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:59.867893 11971 tablet_service.cc:1468] Processing CreateTablet for tablet 5a82c0fcfcbf47d5b88580fc9b2c4474 (DEFAULT_TABLE table=ksck-test-table [id=9823dbf15cb24b85a2b2ba6e85893d4c]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:57:59.868639 11972 tablet_service.cc:1468] Processing CreateTablet for tablet 1af79f5b235142dab0f72e3c69252471 (DEFAULT_TABLE table=ksck-test-table [id=9823dbf15cb24b85a2b2ba6e85893d4c]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:57:59.875921 11972 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1af79f5b235142dab0f72e3c69252471. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:59.877409 11971 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a82c0fcfcbf47d5b88580fc9b2c4474. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:59.914872 11805 tablet_service.cc:1468] Processing CreateTablet for tablet 9bae8bd3195540a4b394e0b851afdb67 (DEFAULT_TABLE table=ksck-test-table [id=9823dbf15cb24b85a2b2ba6e85893d4c]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:57:59.916399 11805 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bae8bd3195540a4b394e0b851afdb67. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:59.930071 11804 tablet_service.cc:1468] Processing CreateTablet for tablet 1af79f5b235142dab0f72e3c69252471 (DEFAULT_TABLE table=ksck-test-table [id=9823dbf15cb24b85a2b2ba6e85893d4c]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:57:59.931424 11804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1af79f5b235142dab0f72e3c69252471. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:59.924799 12030 tablet_bootstrap.cc:492] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P 56bd53feb0344b60852adb4017f906bc: Bootstrap starting.
I20250904 19:57:59.934584 11895 tablet_service.cc:1468] Processing CreateTablet for tablet 9bae8bd3195540a4b394e0b851afdb67 (DEFAULT_TABLE table=ksck-test-table [id=9823dbf15cb24b85a2b2ba6e85893d4c]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:57:59.939633 11895 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bae8bd3195540a4b394e0b851afdb67. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:59.942754 11803 tablet_service.cc:1468] Processing CreateTablet for tablet 5a82c0fcfcbf47d5b88580fc9b2c4474 (DEFAULT_TABLE table=ksck-test-table [id=9823dbf15cb24b85a2b2ba6e85893d4c]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:57:59.944036 11803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a82c0fcfcbf47d5b88580fc9b2c4474. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:59.936589 11893 tablet_service.cc:1468] Processing CreateTablet for tablet 5a82c0fcfcbf47d5b88580fc9b2c4474 (DEFAULT_TABLE table=ksck-test-table [id=9823dbf15cb24b85a2b2ba6e85893d4c]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:57:59.938153 11894 tablet_service.cc:1468] Processing CreateTablet for tablet 1af79f5b235142dab0f72e3c69252471 (DEFAULT_TABLE table=ksck-test-table [id=9823dbf15cb24b85a2b2ba6e85893d4c]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:57:59.948022 11893 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a82c0fcfcbf47d5b88580fc9b2c4474. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:59.948652 11894 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1af79f5b235142dab0f72e3c69252471. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:57:59.976492 12031 tablet_bootstrap.cc:492] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684: Bootstrap starting.
I20250904 19:57:59.984659 12031 tablet_bootstrap.cc:654] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684: Neither blocks nor log segments found. Creating new log.
I20250904 19:57:59.992755 12032 tablet_bootstrap.cc:492] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa: Bootstrap starting.
I20250904 19:57:59.996035 12030 tablet_bootstrap.cc:654] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P 56bd53feb0344b60852adb4017f906bc: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:00.005661 12032 tablet_bootstrap.cc:654] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:00.011269 12031 tablet_bootstrap.cc:492] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684: No bootstrap required, opened a new log
I20250904 19:58:00.011799 12031 ts_tablet_manager.cc:1397] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684: Time spent bootstrapping tablet: real 0.036s user 0.013s sys 0.005s
I20250904 19:58:00.041364 12031 raft_consensus.cc:357] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:00.042243 12031 raft_consensus.cc:383] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:00.042564 12031 raft_consensus.cc:738] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adf3a8303f054fdd83dca3a2b60f0684, State: Initialized, Role: FOLLOWER
I20250904 19:58:00.043452 12031 consensus_queue.cc:260] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:00.044386 12032 tablet_bootstrap.cc:492] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa: No bootstrap required, opened a new log
I20250904 19:58:00.045032 12032 ts_tablet_manager.cc:1397] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa: Time spent bootstrapping tablet: real 0.053s user 0.011s sys 0.006s
I20250904 19:58:00.047920 12032 raft_consensus.cc:357] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:00.048511 12032 raft_consensus.cc:383] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:00.048805 12032 raft_consensus.cc:738] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57a6e99126a4394821e0f0c1e2eb8aa, State: Initialized, Role: FOLLOWER
I20250904 19:58:00.049439 12032 consensus_queue.cc:260] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:00.052234 12032 ts_tablet_manager.cc:1428] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250904 19:58:00.056167 12031 ts_tablet_manager.cc:1428] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684: Time spent starting tablet: real 0.044s user 0.030s sys 0.005s
I20250904 19:58:00.056689 12032 tablet_bootstrap.cc:492] T 9bae8bd3195540a4b394e0b851afdb67 P f57a6e99126a4394821e0f0c1e2eb8aa: Bootstrap starting.
I20250904 19:58:00.059116 12031 tablet_bootstrap.cc:492] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684: Bootstrap starting.
I20250904 19:58:00.063437 12032 tablet_bootstrap.cc:654] T 9bae8bd3195540a4b394e0b851afdb67 P f57a6e99126a4394821e0f0c1e2eb8aa: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:00.066013 12031 tablet_bootstrap.cc:654] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:00.075299 12031 tablet_bootstrap.cc:492] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684: No bootstrap required, opened a new log
I20250904 19:58:00.075313 12032 tablet_bootstrap.cc:492] T 9bae8bd3195540a4b394e0b851afdb67 P f57a6e99126a4394821e0f0c1e2eb8aa: No bootstrap required, opened a new log
I20250904 19:58:00.075964 12031 ts_tablet_manager.cc:1397] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.007s
I20250904 19:58:00.076048 12032 ts_tablet_manager.cc:1397] T 9bae8bd3195540a4b394e0b851afdb67 P f57a6e99126a4394821e0f0c1e2eb8aa: Time spent bootstrapping tablet: real 0.020s user 0.016s sys 0.001s
I20250904 19:58:00.079057 12032 raft_consensus.cc:357] T 9bae8bd3195540a4b394e0b851afdb67 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } }
I20250904 19:58:00.079758 12032 raft_consensus.cc:383] T 9bae8bd3195540a4b394e0b851afdb67 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:00.080053 12032 raft_consensus.cc:738] T 9bae8bd3195540a4b394e0b851afdb67 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57a6e99126a4394821e0f0c1e2eb8aa, State: Initialized, Role: FOLLOWER
I20250904 19:58:00.080669 12032 consensus_queue.cc:260] T 9bae8bd3195540a4b394e0b851afdb67 P f57a6e99126a4394821e0f0c1e2eb8aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } }
I20250904 19:58:00.080756 12031 raft_consensus.cc:357] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } }
I20250904 19:58:00.081527 12031 raft_consensus.cc:383] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:00.081820 12031 raft_consensus.cc:738] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adf3a8303f054fdd83dca3a2b60f0684, State: Initialized, Role: FOLLOWER
I20250904 19:58:00.082893 12032 ts_tablet_manager.cc:1428] T 9bae8bd3195540a4b394e0b851afdb67 P f57a6e99126a4394821e0f0c1e2eb8aa: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250904 19:58:00.082552 12031 consensus_queue.cc:260] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } }
I20250904 19:58:00.083982 12032 tablet_bootstrap.cc:492] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa: Bootstrap starting.
I20250904 19:58:00.087049 12031 ts_tablet_manager.cc:1428] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20250904 19:58:00.088063 12031 tablet_bootstrap.cc:492] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P adf3a8303f054fdd83dca3a2b60f0684: Bootstrap starting.
I20250904 19:58:00.090304 12032 tablet_bootstrap.cc:654] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:00.094117 12031 tablet_bootstrap.cc:654] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P adf3a8303f054fdd83dca3a2b60f0684: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:00.116477 12031 tablet_bootstrap.cc:492] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P adf3a8303f054fdd83dca3a2b60f0684: No bootstrap required, opened a new log
I20250904 19:58:00.116959 12031 ts_tablet_manager.cc:1397] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P adf3a8303f054fdd83dca3a2b60f0684: Time spent bootstrapping tablet: real 0.029s user 0.010s sys 0.012s
I20250904 19:58:00.118358 12032 tablet_bootstrap.cc:492] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa: No bootstrap required, opened a new log
I20250904 19:58:00.119616 12032 ts_tablet_manager.cc:1397] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa: Time spent bootstrapping tablet: real 0.036s user 0.013s sys 0.020s
I20250904 19:58:00.119738 12031 raft_consensus.cc:357] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:00.120574 12031 raft_consensus.cc:383] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:00.120882 12031 raft_consensus.cc:738] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adf3a8303f054fdd83dca3a2b60f0684, State: Initialized, Role: FOLLOWER
I20250904 19:58:00.122344 12032 raft_consensus.cc:357] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:00.123109 12032 raft_consensus.cc:383] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:00.121765 12031 consensus_queue.cc:260] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P adf3a8303f054fdd83dca3a2b60f0684 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:00.123417 12032 raft_consensus.cc:738] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f57a6e99126a4394821e0f0c1e2eb8aa, State: Initialized, Role: FOLLOWER
I20250904 19:58:00.127154 12032 consensus_queue.cc:260] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:00.132073 11755 catalog_manager.cc:1261] Loaded cluster ID: 96c9aabb12c346dfae562abcf50e9800
I20250904 19:58:00.132442 11755 catalog_manager.cc:1554] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3: loading cluster ID for follower catalog manager: success
I20250904 19:58:00.133689 11750 catalog_manager.cc:1261] Loaded cluster ID: 96c9aabb12c346dfae562abcf50e9800
I20250904 19:58:00.134388 12030 tablet_bootstrap.cc:492] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P 56bd53feb0344b60852adb4017f906bc: No bootstrap required, opened a new log
I20250904 19:58:00.130708 12032 ts_tablet_manager.cc:1428] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20250904 19:58:00.137902 12031 ts_tablet_manager.cc:1428] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P adf3a8303f054fdd83dca3a2b60f0684: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20250904 19:58:00.146446 12030 ts_tablet_manager.cc:1397] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P 56bd53feb0344b60852adb4017f906bc: Time spent bootstrapping tablet: real 0.222s user 0.011s sys 0.007s
I20250904 19:58:00.140149 11755 catalog_manager.cc:1576] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3: acquiring CA information for follower catalog manager: success
I20250904 19:58:00.145969 11750 catalog_manager.cc:1554] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa: loading cluster ID for follower catalog manager: success
I20250904 19:58:00.153846 11755 catalog_manager.cc:1604] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:58:00.150692 12030 raft_consensus.cc:357] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P 56bd53feb0344b60852adb4017f906bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:00.157169 12030 raft_consensus.cc:383] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P 56bd53feb0344b60852adb4017f906bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:00.157495 12030 raft_consensus.cc:738] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P 56bd53feb0344b60852adb4017f906bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56bd53feb0344b60852adb4017f906bc, State: Initialized, Role: FOLLOWER
I20250904 19:58:00.158149 12030 consensus_queue.cc:260] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P 56bd53feb0344b60852adb4017f906bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:00.161691 12030 ts_tablet_manager.cc:1428] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P 56bd53feb0344b60852adb4017f906bc: Time spent starting tablet: real 0.014s user 0.003s sys 0.002s
I20250904 19:58:00.162719 12030 tablet_bootstrap.cc:492] T 9bae8bd3195540a4b394e0b851afdb67 P 56bd53feb0344b60852adb4017f906bc: Bootstrap starting.
I20250904 19:58:00.168578 12030 tablet_bootstrap.cc:654] T 9bae8bd3195540a4b394e0b851afdb67 P 56bd53feb0344b60852adb4017f906bc: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:00.159411 11750 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa: acquiring CA information for follower catalog manager: success
I20250904 19:58:00.186743 12030 tablet_bootstrap.cc:492] T 9bae8bd3195540a4b394e0b851afdb67 P 56bd53feb0344b60852adb4017f906bc: No bootstrap required, opened a new log
I20250904 19:58:00.187341 12030 ts_tablet_manager.cc:1397] T 9bae8bd3195540a4b394e0b851afdb67 P 56bd53feb0344b60852adb4017f906bc: Time spent bootstrapping tablet: real 0.025s user 0.016s sys 0.008s
I20250904 19:58:00.190043 12030 raft_consensus.cc:357] T 9bae8bd3195540a4b394e0b851afdb67 P 56bd53feb0344b60852adb4017f906bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } }
I20250904 19:58:00.190800 12030 raft_consensus.cc:383] T 9bae8bd3195540a4b394e0b851afdb67 P 56bd53feb0344b60852adb4017f906bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:00.191041 12030 raft_consensus.cc:738] T 9bae8bd3195540a4b394e0b851afdb67 P 56bd53feb0344b60852adb4017f906bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56bd53feb0344b60852adb4017f906bc, State: Initialized, Role: FOLLOWER
I20250904 19:58:00.191694 12030 consensus_queue.cc:260] T 9bae8bd3195540a4b394e0b851afdb67 P 56bd53feb0344b60852adb4017f906bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } }
I20250904 19:58:00.193645 12030 ts_tablet_manager.cc:1428] T 9bae8bd3195540a4b394e0b851afdb67 P 56bd53feb0344b60852adb4017f906bc: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250904 19:58:00.194630 12030 tablet_bootstrap.cc:492] T 1af79f5b235142dab0f72e3c69252471 P 56bd53feb0344b60852adb4017f906bc: Bootstrap starting.
I20250904 19:58:00.200927 12030 tablet_bootstrap.cc:654] T 1af79f5b235142dab0f72e3c69252471 P 56bd53feb0344b60852adb4017f906bc: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:00.201699 11750 catalog_manager.cc:1604] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:58:00.212594 12030 tablet_bootstrap.cc:492] T 1af79f5b235142dab0f72e3c69252471 P 56bd53feb0344b60852adb4017f906bc: No bootstrap required, opened a new log
I20250904 19:58:00.213099 12030 ts_tablet_manager.cc:1397] T 1af79f5b235142dab0f72e3c69252471 P 56bd53feb0344b60852adb4017f906bc: Time spent bootstrapping tablet: real 0.019s user 0.015s sys 0.001s
I20250904 19:58:00.216940 12030 raft_consensus.cc:357] T 1af79f5b235142dab0f72e3c69252471 P 56bd53feb0344b60852adb4017f906bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:00.217617 12030 raft_consensus.cc:383] T 1af79f5b235142dab0f72e3c69252471 P 56bd53feb0344b60852adb4017f906bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:00.217969 12030 raft_consensus.cc:738] T 1af79f5b235142dab0f72e3c69252471 P 56bd53feb0344b60852adb4017f906bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56bd53feb0344b60852adb4017f906bc, State: Initialized, Role: FOLLOWER
I20250904 19:58:00.220752 12030 consensus_queue.cc:260] T 1af79f5b235142dab0f72e3c69252471 P 56bd53feb0344b60852adb4017f906bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:00.223003 12030 ts_tablet_manager.cc:1428] T 1af79f5b235142dab0f72e3c69252471 P 56bd53feb0344b60852adb4017f906bc: Time spent starting tablet: real 0.010s user 0.003s sys 0.002s
I20250904 19:58:00.285109 12037 raft_consensus.cc:491] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:00.285661 12037 raft_consensus.cc:513] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:00.287921 12037 leader_election.cc:290] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129:41455), 56bd53feb0344b60852adb4017f906bc (127.10.24.131:37237)
I20250904 19:58:00.292905 12036 raft_consensus.cc:491] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:00.293424 12036 raft_consensus.cc:513] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } }
I20250904 19:58:00.300045 11815 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5a82c0fcfcbf47d5b88580fc9b2c4474" candidate_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf3a8303f054fdd83dca3a2b60f0684" is_pre_election: true
I20250904 19:58:00.301090 11815 raft_consensus.cc:2466] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57a6e99126a4394821e0f0c1e2eb8aa in term 0.
I20250904 19:58:00.300153 12036 leader_election.cc:290] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56bd53feb0344b60852adb4017f906bc (127.10.24.131:37237), f57a6e99126a4394821e0f0c1e2eb8aa (127.10.24.130:44159)
I20250904 19:58:00.304050 11859 leader_election.cc:304] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adf3a8303f054fdd83dca3a2b60f0684, f57a6e99126a4394821e0f0c1e2eb8aa; no voters:
I20250904 19:58:00.304989 12037 raft_consensus.cc:2802] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:00.305373 12037 raft_consensus.cc:491] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:00.305681 12037 raft_consensus.cc:3058] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:00.314775 11982 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "9bae8bd3195540a4b394e0b851afdb67" candidate_uuid: "adf3a8303f054fdd83dca3a2b60f0684" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd53feb0344b60852adb4017f906bc" is_pre_election: true
I20250904 19:58:00.315517 11982 raft_consensus.cc:2466] T 9bae8bd3195540a4b394e0b851afdb67 P 56bd53feb0344b60852adb4017f906bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adf3a8303f054fdd83dca3a2b60f0684 in term 0.
I20250904 19:58:00.317006 11778 leader_election.cc:304] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56bd53feb0344b60852adb4017f906bc, adf3a8303f054fdd83dca3a2b60f0684; no voters:
I20250904 19:58:00.317865 12036 raft_consensus.cc:2802] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:00.318207 12036 raft_consensus.cc:491] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:00.318508 12036 raft_consensus.cc:3058] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:00.318977 12037 raft_consensus.cc:513] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:00.322265 11815 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5a82c0fcfcbf47d5b88580fc9b2c4474" candidate_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf3a8303f054fdd83dca3a2b60f0684"
I20250904 19:58:00.322796 11815 raft_consensus.cc:3058] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:00.324153 12037 leader_election.cc:290] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [CANDIDATE]: Term 1 election: Requested vote from peers adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129:41455), 56bd53feb0344b60852adb4017f906bc (127.10.24.131:37237)
I20250904 19:58:00.326325 12036 raft_consensus.cc:513] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } }
I20250904 19:58:00.329217 12036 leader_election.cc:290] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [CANDIDATE]: Term 1 election: Requested vote from peers 56bd53feb0344b60852adb4017f906bc (127.10.24.131:37237), f57a6e99126a4394821e0f0c1e2eb8aa (127.10.24.130:44159)
I20250904 19:58:00.329533 11905 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "9bae8bd3195540a4b394e0b851afdb67" candidate_uuid: "adf3a8303f054fdd83dca3a2b60f0684" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" is_pre_election: true
I20250904 19:58:00.330216 11905 raft_consensus.cc:2466] T 9bae8bd3195540a4b394e0b851afdb67 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adf3a8303f054fdd83dca3a2b60f0684 in term 0.
I20250904 19:58:00.329421 11815 raft_consensus.cc:2466] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P adf3a8303f054fdd83dca3a2b60f0684 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57a6e99126a4394821e0f0c1e2eb8aa in term 1.
I20250904 19:58:00.330246 11904 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "9bae8bd3195540a4b394e0b851afdb67" candidate_uuid: "adf3a8303f054fdd83dca3a2b60f0684" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa"
I20250904 19:58:00.332732 11904 raft_consensus.cc:3058] T 9bae8bd3195540a4b394e0b851afdb67 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:00.332949 11859 leader_election.cc:304] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: adf3a8303f054fdd83dca3a2b60f0684, f57a6e99126a4394821e0f0c1e2eb8aa; no voters:
I20250904 19:58:00.333050 11982 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "9bae8bd3195540a4b394e0b851afdb67" candidate_uuid: "adf3a8303f054fdd83dca3a2b60f0684" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd53feb0344b60852adb4017f906bc"
I20250904 19:58:00.333623 11982 raft_consensus.cc:3058] T 9bae8bd3195540a4b394e0b851afdb67 P 56bd53feb0344b60852adb4017f906bc [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:00.335706 11983 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5a82c0fcfcbf47d5b88580fc9b2c4474" candidate_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd53feb0344b60852adb4017f906bc" is_pre_election: true
I20250904 19:58:00.336010 11981 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5a82c0fcfcbf47d5b88580fc9b2c4474" candidate_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd53feb0344b60852adb4017f906bc"
I20250904 19:58:00.336647 11981 raft_consensus.cc:3058] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P 56bd53feb0344b60852adb4017f906bc [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:00.338464 12037 raft_consensus.cc:2802] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:00.340154 11982 raft_consensus.cc:2466] T 9bae8bd3195540a4b394e0b851afdb67 P 56bd53feb0344b60852adb4017f906bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adf3a8303f054fdd83dca3a2b60f0684 in term 1.
I20250904 19:58:00.342427 12036 raft_consensus.cc:491] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:00.342893 12036 raft_consensus.cc:513] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:00.344808 12036 leader_election.cc:290] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f57a6e99126a4394821e0f0c1e2eb8aa (127.10.24.130:44159), 56bd53feb0344b60852adb4017f906bc (127.10.24.131:37237)
I20250904 19:58:00.348264 11778 leader_election.cc:304] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56bd53feb0344b60852adb4017f906bc, adf3a8303f054fdd83dca3a2b60f0684; no voters:
I20250904 19:58:00.348743 12050 raft_consensus.cc:491] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:00.349170 12050 raft_consensus.cc:513] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:00.349817 12036 raft_consensus.cc:2802] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:00.351198 12050 leader_election.cc:290] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129:41455), 56bd53feb0344b60852adb4017f906bc (127.10.24.131:37237)
I20250904 19:58:00.351933 11982 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1af79f5b235142dab0f72e3c69252471" candidate_uuid: "adf3a8303f054fdd83dca3a2b60f0684" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd53feb0344b60852adb4017f906bc" is_pre_election: true
I20250904 19:58:00.352631 11982 raft_consensus.cc:2466] T 1af79f5b235142dab0f72e3c69252471 P 56bd53feb0344b60852adb4017f906bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate adf3a8303f054fdd83dca3a2b60f0684 in term 0.
I20250904 19:58:00.353957 11982 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1af79f5b235142dab0f72e3c69252471" candidate_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd53feb0344b60852adb4017f906bc" is_pre_election: true
I20250904 19:58:00.354518 11982 raft_consensus.cc:2466] T 1af79f5b235142dab0f72e3c69252471 P 56bd53feb0344b60852adb4017f906bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57a6e99126a4394821e0f0c1e2eb8aa in term 0.
I20250904 19:58:00.355585 11778 leader_election.cc:304] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56bd53feb0344b60852adb4017f906bc, adf3a8303f054fdd83dca3a2b60f0684; no voters:
I20250904 19:58:00.356215 11981 raft_consensus.cc:2466] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P 56bd53feb0344b60852adb4017f906bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57a6e99126a4394821e0f0c1e2eb8aa in term 1.
I20250904 19:58:00.356698 11815 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1af79f5b235142dab0f72e3c69252471" candidate_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf3a8303f054fdd83dca3a2b60f0684" is_pre_election: true
I20250904 19:58:00.356840 11856 leader_election.cc:304] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56bd53feb0344b60852adb4017f906bc, f57a6e99126a4394821e0f0c1e2eb8aa; no voters:
I20250904 19:58:00.357374 11815 raft_consensus.cc:2466] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f57a6e99126a4394821e0f0c1e2eb8aa in term 0.
I20250904 19:58:00.357823 12051 raft_consensus.cc:2802] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:00.358285 12051 raft_consensus.cc:491] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:00.358646 12051 raft_consensus.cc:3058] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:00.358640 11905 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1af79f5b235142dab0f72e3c69252471" candidate_uuid: "adf3a8303f054fdd83dca3a2b60f0684" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" is_pre_election: true
I20250904 19:58:00.359722 11904 raft_consensus.cc:2466] T 9bae8bd3195540a4b394e0b851afdb67 P f57a6e99126a4394821e0f0c1e2eb8aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate adf3a8303f054fdd83dca3a2b60f0684 in term 1.
I20250904 19:58:00.364192 12053 raft_consensus.cc:2802] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:00.371421 12037 raft_consensus.cc:695] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [term 1 LEADER]: Becoming Leader. State: Replica: f57a6e99126a4394821e0f0c1e2eb8aa, State: Running, Role: LEADER
I20250904 19:58:00.372304 12036 raft_consensus.cc:695] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [term 1 LEADER]: Becoming Leader. State: Replica: adf3a8303f054fdd83dca3a2b60f0684, State: Running, Role: LEADER
I20250904 19:58:00.372512 12053 raft_consensus.cc:491] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:00.372931 12053 raft_consensus.cc:3058] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:00.365561 12051 raft_consensus.cc:513] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:00.373238 12036 consensus_queue.cc:237] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } }
I20250904 19:58:00.374208 11905 raft_consensus.cc:2391] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate adf3a8303f054fdd83dca3a2b60f0684 in current term 1: Already voted for candidate f57a6e99126a4394821e0f0c1e2eb8aa in this term.
I20250904 19:58:00.373303 12037 consensus_queue.cc:237] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:00.379839 11815 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1af79f5b235142dab0f72e3c69252471" candidate_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adf3a8303f054fdd83dca3a2b60f0684"
I20250904 19:58:00.380100 11981 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1af79f5b235142dab0f72e3c69252471" candidate_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd53feb0344b60852adb4017f906bc"
I20250904 19:58:00.382136 11981 raft_consensus.cc:3058] T 1af79f5b235142dab0f72e3c69252471 P 56bd53feb0344b60852adb4017f906bc [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:00.379900 12051 leader_election.cc:290] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [CANDIDATE]: Term 1 election: Requested vote from peers adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129:41455), 56bd53feb0344b60852adb4017f906bc (127.10.24.131:37237)
I20250904 19:58:00.387420 12053 raft_consensus.cc:513] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:00.388609 11815 raft_consensus.cc:2391] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f57a6e99126a4394821e0f0c1e2eb8aa in current term 1: Already voted for candidate adf3a8303f054fdd83dca3a2b60f0684 in this term.
I20250904 19:58:00.390883 11905 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1af79f5b235142dab0f72e3c69252471" candidate_uuid: "adf3a8303f054fdd83dca3a2b60f0684" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa"
I20250904 19:58:00.391712 11905 raft_consensus.cc:2391] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate adf3a8303f054fdd83dca3a2b60f0684 in current term 1: Already voted for candidate f57a6e99126a4394821e0f0c1e2eb8aa in this term.
I20250904 19:58:00.395587 12053 leader_election.cc:290] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684 [CANDIDATE]: Term 1 election: Requested vote from peers f57a6e99126a4394821e0f0c1e2eb8aa (127.10.24.130:44159), 56bd53feb0344b60852adb4017f906bc (127.10.24.131:37237)
I20250904 19:58:00.399622 11982 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1af79f5b235142dab0f72e3c69252471" candidate_uuid: "adf3a8303f054fdd83dca3a2b60f0684" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56bd53feb0344b60852adb4017f906bc"
I20250904 19:58:00.400415 11558 catalog_manager.cc:5582] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa reported cstate change: term changed from 0 to 1, leader changed from <none> to f57a6e99126a4394821e0f0c1e2eb8aa (127.10.24.130). New cstate: current_term: 1 leader_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } health_report { overall_health: UNKNOWN } } }
I20250904 19:58:00.401180 11778 leader_election.cc:304] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: adf3a8303f054fdd83dca3a2b60f0684; no voters: 56bd53feb0344b60852adb4017f906bc, f57a6e99126a4394821e0f0c1e2eb8aa
I20250904 19:58:00.406883 12036 raft_consensus.cc:2747] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250904 19:58:00.403012 11557 catalog_manager.cc:5582] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 reported cstate change: term changed from 0 to 1, leader changed from <none> to adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129). New cstate: current_term: 1 leader_uuid: "adf3a8303f054fdd83dca3a2b60f0684" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } health_report { overall_health: HEALTHY } } }
I20250904 19:58:00.402276 11981 raft_consensus.cc:2466] T 1af79f5b235142dab0f72e3c69252471 P 56bd53feb0344b60852adb4017f906bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f57a6e99126a4394821e0f0c1e2eb8aa in term 1.
I20250904 19:58:00.411202 11856 leader_election.cc:304] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 56bd53feb0344b60852adb4017f906bc, f57a6e99126a4394821e0f0c1e2eb8aa; no voters: adf3a8303f054fdd83dca3a2b60f0684
I20250904 19:58:00.412093 12037 raft_consensus.cc:2802] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:00.412567 12037 raft_consensus.cc:695] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [term 1 LEADER]: Becoming Leader. State: Replica: f57a6e99126a4394821e0f0c1e2eb8aa, State: Running, Role: LEADER
I20250904 19:58:00.413360 12037 consensus_queue.cc:237] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:00.484683 11558 catalog_manager.cc:5582] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa reported cstate change: term changed from 0 to 1, leader changed from <none> to f57a6e99126a4394821e0f0c1e2eb8aa (127.10.24.130). New cstate: current_term: 1 leader_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } health_report { overall_health: UNKNOWN } } }
I20250904 19:58:00.798003 12037 consensus_queue.cc:1035] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20250904 19:58:00.849637 12050 consensus_queue.cc:1035] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:00.880587 12036 consensus_queue.cc:1035] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [LEADER]: Connected to new peer: Peer: permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:00.904042 12050 consensus_queue.cc:1035] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 41455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:00.929893 12036 consensus_queue.cc:1035] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:00.938191 12050 consensus_queue.cc:1035] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:01.234961 10338 tablet_server.cc:178] TabletServer@127.10.24.129:0 shutting down...
W20250904 19:58:01.260970 11859 consensus_peers.cc:489] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa -> Peer adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129:41455): Couldn't send request to peer adf3a8303f054fdd83dca3a2b60f0684. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250904 19:58:01.290532 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:01.292073 10338 tablet_replica.cc:331] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P adf3a8303f054fdd83dca3a2b60f0684: stopping tablet replica
I20250904 19:58:01.292891 10338 raft_consensus.cc:2241] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P adf3a8303f054fdd83dca3a2b60f0684 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:01.293502 10338 raft_consensus.cc:2270] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P adf3a8303f054fdd83dca3a2b60f0684 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:01.296002 10338 tablet_replica.cc:331] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684: stopping tablet replica
I20250904 19:58:01.296474 10338 raft_consensus.cc:2241] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:01.296836 10338 raft_consensus.cc:2270] T 1af79f5b235142dab0f72e3c69252471 P adf3a8303f054fdd83dca3a2b60f0684 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:01.299039 10338 tablet_replica.cc:331] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684: stopping tablet replica
I20250904 19:58:01.299702 10338 raft_consensus.cc:2241] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:01.300669 10338 raft_consensus.cc:2270] T 9bae8bd3195540a4b394e0b851afdb67 P adf3a8303f054fdd83dca3a2b60f0684 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:01.327116 10338 tablet_server.cc:195] TabletServer@127.10.24.129:0 shutdown complete.
I20250904 19:58:01.341823 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:01.349308 12110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:01.348125 12109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:01.354848 12112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:01.355434 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:01.357197 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:01.357461 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:01.357658 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015881357642 us; error 0 us; skew 500 ppm
I20250904 19:58:01.358275 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:01.360869 10338 webserver.cc:480] Webserver started at http://127.10.24.129:42853/ using document root <none> and password file <none>
I20250904 19:58:01.361307 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:01.361465 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:01.361719 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:01.362751 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/ts-0-root2/instance:
uuid: "a81a77f1da8247b989790c2626caa6bd"
format_stamp: "Formatted at 2025-09-04 19:58:01 on dist-test-slave-f3xj"
I20250904 19:58:01.370744 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.007s sys 0.002s
I20250904 19:58:01.374275 12117 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:01.374959 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250904 19:58:01.375255 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/ts-0-root2
uuid: "a81a77f1da8247b989790c2626caa6bd"
format_stamp: "Formatted at 2025-09-04 19:58:01 on dist-test-slave-f3xj"
I20250904 19:58:01.375514 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/ts-0-root2
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/ts-0-root2
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTabletServerMismatchedUUID.1757015869512910-10338-0/minicluster-data/ts-0-root2/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:01.400493 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:01.401621 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:01.403339 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:01.405742 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:01.405942 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:01.406154 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:01.406303 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
W20250904 19:58:01.424626 11859 consensus_peers.cc:489] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa -> Peer adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129:41455): Couldn't send request to peer adf3a8303f054fdd83dca3a2b60f0684. Status: Network error: Client connection negotiation failed: client connection to 127.10.24.129:41455: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 19:58:01.478125 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.129:41455
I20250904 19:58:01.478231 12180 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.129:41455 every 8 connection(s)
W20250904 19:58:01.494621 12181 heartbeater.cc:646] Failed to heartbeat to 127.0.0.1:7051 (0 consecutive failures): Network error: Failed to ping master at 127.0.0.1:7051: Client connection negotiation failed: client connection to 127.0.0.1:7051: connect: Connection refused (error 111)
W20250904 19:58:01.501982 12181 heartbeater.cc:412] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20250904 19:58:01.542815 11972 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250904 19:58:01.552601 11893 tablet_service.cc:1430] Tablet server has 2 leaders and 0 scanners
W20250904 19:58:01.811785 12156 tablet_service.cc:374] Invalid argument: UpdateConsensus: Wrong destination UUID requested. Local UUID: a81a77f1da8247b989790c2626caa6bd. Requested UUID: adf3a8303f054fdd83dca3a2b60f0684: from {username='slave'} at 127.0.0.1:43908: tablet_id: "5a82c0fcfcbf47d5b88580fc9b2c4474" caller_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" caller_term: 1 preceding_id { term: 1 index: 1 } dest_uuid: "adf3a8303f054fdd83dca3a2b60f0684" committed_index: 1 all_replicated_index: 1 safe_timestamp: 7196737051848331264 last_idx_appended_to_leader: 1
I20250904 19:58:01.813654 11859 consensus_queue.cc:579] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [LEADER]: The tablet replica hosted on peer adf3a8303f054fdd83dca3a2b60f0684 has failed
I20250904 19:58:01.827917 11905 consensus_queue.cc:237] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:01.836853 11981 raft_consensus.cc:1273] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P 56bd53feb0344b60852adb4017f906bc [term 1 FOLLOWER]: Refusing update from remote peer f57a6e99126a4394821e0f0c1e2eb8aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:01.838263 12051 consensus_queue.cc:1035] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250904 19:58:01.851801 12050 raft_consensus.cc:2953] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } } }
I20250904 19:58:01.859898 11981 raft_consensus.cc:2953] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P 56bd53feb0344b60852adb4017f906bc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } } }
I20250904 19:58:01.868707 11545 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet 5a82c0fcfcbf47d5b88580fc9b2c4474 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20250904 19:58:01.873447 12156 tablet_service.cc:374] Invalid argument: UpdateConsensus: Wrong destination UUID requested. Local UUID: a81a77f1da8247b989790c2626caa6bd. Requested UUID: adf3a8303f054fdd83dca3a2b60f0684: from {username='slave'} at 127.0.0.1:43908: tablet_id: "1af79f5b235142dab0f72e3c69252471" caller_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" caller_term: 1 preceding_id { term: 1 index: 1 } dest_uuid: "adf3a8303f054fdd83dca3a2b60f0684" committed_index: 1 all_replicated_index: 1 safe_timestamp: 7196737052145377280 last_idx_appended_to_leader: 1
I20250904 19:58:01.877074 11859 consensus_queue.cc:579] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [LEADER]: The tablet replica hosted on peer adf3a8303f054fdd83dca3a2b60f0684 has failed
I20250904 19:58:01.884752 11560 catalog_manager.cc:5582] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa reported cstate change: config changed from index -1 to 2, VOTER adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129) evicted. New cstate: current_term: 1 leader_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } health_report { overall_health: HEALTHY } } }
I20250904 19:58:01.885846 10338 tablet_server.cc:178] TabletServer@127.10.24.129:41455 shutting down...
I20250904 19:58:01.966683 11905 consensus_queue.cc:237] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } } peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: NON_VOTER last_known_addr { host: "127.10.24.129" port: 41455 } attrs { promote: true } }
I20250904 19:58:01.984354 11981 raft_consensus.cc:1273] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P 56bd53feb0344b60852adb4017f906bc [term 1 FOLLOWER]: Refusing update from remote peer f57a6e99126a4394821e0f0c1e2eb8aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20250904 19:58:01.987960 11859 consensus_peers.cc:489] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa -> Peer adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129:41455): Couldn't send request to peer adf3a8303f054fdd83dca3a2b60f0684. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250904 19:58:01.988153 12037 consensus_queue.cc:1035] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20250904 19:58:01.989187 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:01.996320 11905 consensus_queue.cc:237] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } }
I20250904 19:58:01.998405 12037 raft_consensus.cc:2953] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } } peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: NON_VOTER last_known_addr { host: "127.10.24.129" port: 41455 } attrs { promote: true } } }
I20250904 19:58:02.003032 11983 raft_consensus.cc:1273] T 1af79f5b235142dab0f72e3c69252471 P 56bd53feb0344b60852adb4017f906bc [term 1 FOLLOWER]: Refusing update from remote peer f57a6e99126a4394821e0f0c1e2eb8aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:02.002405 11981 raft_consensus.cc:2953] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P 56bd53feb0344b60852adb4017f906bc [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, NON_VOTER adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } } peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: NON_VOTER last_known_addr { host: "127.10.24.129" port: 41455 } attrs { promote: true } } }
I20250904 19:58:02.004540 12050 consensus_queue.cc:1035] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20250904 19:58:02.004899 12119 connection.cc:441] server connection from 127.0.0.1:43922 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43922 (request call id 0) could send its response
I20250904 19:58:02.022048 10338 tablet_server.cc:195] TabletServer@127.10.24.129:41455 shutdown complete.
I20250904 19:58:02.031847 10338 tablet_server.cc:178] TabletServer@127.10.24.130:0 shutting down...
I20250904 19:58:02.036288 11545 catalog_manager.cc:5095] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 5a82c0fcfcbf47d5b88580fc9b2c4474 with cas_config_opid_index 2: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
W20250904 19:58:02.043655 11547 catalog_manager.cc:4671] TS adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129:41455): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 5a82c0fcfcbf47d5b88580fc9b2c4474: Network error: Client connection negotiation failed: client connection to 127.10.24.129:41455: connect: Connection refused (error 111)
I20250904 19:58:02.046666 11558 catalog_manager.cc:5582] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P 56bd53feb0344b60852adb4017f906bc reported cstate change: config changed from index 2 to 3, NON_VOTER adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129) added. New cstate: current_term: 1 leader_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } } peers { permanent_uuid: "adf3a8303f054fdd83dca3a2b60f0684" member_type: NON_VOTER last_known_addr { host: "127.10.24.129" port: 41455 } attrs { promote: true } } }
I20250904 19:58:02.048518 12050 raft_consensus.cc:2953] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } } }
I20250904 19:58:02.053586 11983 raft_consensus.cc:2953] T 1af79f5b235142dab0f72e3c69252471 P 56bd53feb0344b60852adb4017f906bc [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, VOTER adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129) evicted. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } } }
I20250904 19:58:02.076154 11560 catalog_manager.cc:5582] T 1af79f5b235142dab0f72e3c69252471 P 56bd53feb0344b60852adb4017f906bc reported cstate change: config changed from index -1 to 2, VOTER adf3a8303f054fdd83dca3a2b60f0684 (127.10.24.129) evicted. New cstate: current_term: 1 leader_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "f57a6e99126a4394821e0f0c1e2eb8aa" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 44159 } } peers { permanent_uuid: "56bd53feb0344b60852adb4017f906bc" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37237 } } }
I20250904 19:58:02.079351 11545 catalog_manager.cc:5095] ChangeConfig:REMOVE_PEER RPC for tablet 1af79f5b235142dab0f72e3c69252471 with cas_config_opid_index -1: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20250904 19:58:02.117785 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:02.118371 10338 tablet_replica.cc:331] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa: stopping tablet replica
I20250904 19:58:02.118886 10338 raft_consensus.cc:2241] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:02.119750 10338 raft_consensus.cc:2270] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P f57a6e99126a4394821e0f0c1e2eb8aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:02.121827 10338 tablet_replica.cc:331] T 9bae8bd3195540a4b394e0b851afdb67 P f57a6e99126a4394821e0f0c1e2eb8aa: stopping tablet replica
I20250904 19:58:02.122284 10338 raft_consensus.cc:2241] T 9bae8bd3195540a4b394e0b851afdb67 P f57a6e99126a4394821e0f0c1e2eb8aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:02.122641 10338 raft_consensus.cc:2270] T 9bae8bd3195540a4b394e0b851afdb67 P f57a6e99126a4394821e0f0c1e2eb8aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:02.124111 10338 tablet_replica.cc:331] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa: stopping tablet replica
I20250904 19:58:02.124511 10338 raft_consensus.cc:2241] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:02.125063 10338 raft_consensus.cc:2270] T 1af79f5b235142dab0f72e3c69252471 P f57a6e99126a4394821e0f0c1e2eb8aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:02.150111 10338 tablet_server.cc:195] TabletServer@127.10.24.130:0 shutdown complete.
I20250904 19:58:02.161887 10338 tablet_server.cc:178] TabletServer@127.10.24.131:0 shutting down...
I20250904 19:58:02.185492 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:02.186093 10338 tablet_replica.cc:331] T 1af79f5b235142dab0f72e3c69252471 P 56bd53feb0344b60852adb4017f906bc: stopping tablet replica
I20250904 19:58:02.186609 10338 raft_consensus.cc:2241] T 1af79f5b235142dab0f72e3c69252471 P 56bd53feb0344b60852adb4017f906bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:02.187053 10338 raft_consensus.cc:2270] T 1af79f5b235142dab0f72e3c69252471 P 56bd53feb0344b60852adb4017f906bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:02.188968 10338 tablet_replica.cc:331] T 9bae8bd3195540a4b394e0b851afdb67 P 56bd53feb0344b60852adb4017f906bc: stopping tablet replica
I20250904 19:58:02.189425 10338 raft_consensus.cc:2241] T 9bae8bd3195540a4b394e0b851afdb67 P 56bd53feb0344b60852adb4017f906bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:02.189774 10338 raft_consensus.cc:2270] T 9bae8bd3195540a4b394e0b851afdb67 P 56bd53feb0344b60852adb4017f906bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:02.191232 10338 tablet_replica.cc:331] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P 56bd53feb0344b60852adb4017f906bc: stopping tablet replica
I20250904 19:58:02.191668 10338 raft_consensus.cc:2241] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P 56bd53feb0344b60852adb4017f906bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:02.192097 10338 raft_consensus.cc:2270] T 5a82c0fcfcbf47d5b88580fc9b2c4474 P 56bd53feb0344b60852adb4017f906bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:02.211349 10338 tablet_server.cc:195] TabletServer@127.10.24.131:0 shutdown complete.
I20250904 19:58:02.223938 10338 master.cc:561] Master@127.10.24.190:45327 shutting down...
I20250904 19:58:02.406438 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:02.407302 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:02.407675 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 5e7da9c2711646749a203ea10baf03a9: stopping tablet replica
I20250904 19:58:02.427817 10338 master.cc:583] Master@127.10.24.190:45327 shutdown complete.
I20250904 19:58:02.441196 10338 master.cc:561] Master@127.10.24.189:40465 shutting down...
I20250904 19:58:02.456496 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:02.456960 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:02.457237 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P c0ee841362264b408de443e2f2eae8f3: stopping tablet replica
I20250904 19:58:02.467199 10338 master.cc:583] Master@127.10.24.189:40465 shutdown complete.
I20250904 19:58:02.478590 10338 master.cc:561] Master@127.10.24.188:41645 shutting down...
I20250904 19:58:02.493418 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:02.493891 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:02.494192 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 082185d1cb27408b8bf75c9b69857cfa: stopping tablet replica
I20250904 19:58:02.512624 10338 master.cc:583] Master@127.10.24.188:41645 shutdown complete.
[ OK ] RemoteKsckTest.TestTabletServerMismatchedUUID (3734 ms)
[ RUN ] RemoteKsckTest.TestTableConsistency
I20250904 19:58:02.541587 10338 test_util.cc:276] Using random seed: 596322814
I20250904 19:58:02.542485 10338 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.24.190:34335,127.10.24.189:36735,127.10.24.188:34959
I20250904 19:58:02.543715 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:02.548163 12212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:02.548301 12213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:02.549599 12215 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:02.550354 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:02.551091 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:02.551265 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:02.551399 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015882551382 us; error 0 us; skew 500 ppm
I20250904 19:58:02.552043 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:02.554231 10338 webserver.cc:480] Webserver started at http://127.10.24.190:38159/ using document root <none> and password file <none>
I20250904 19:58:02.554667 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:02.554829 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:02.555050 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:02.556066 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/master-0-root/instance:
uuid: "163c716beb0746a3a914158f4ad2b2de"
format_stamp: "Formatted at 2025-09-04 19:58:02 on dist-test-slave-f3xj"
I20250904 19:58:02.559927 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20250904 19:58:02.562798 12220 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:02.563412 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250904 19:58:02.563679 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/master-0-root
uuid: "163c716beb0746a3a914158f4ad2b2de"
format_stamp: "Formatted at 2025-09-04 19:58:02 on dist-test-slave-f3xj"
I20250904 19:58:02.563920 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:02.585800 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:02.586875 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:02.621388 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.190:34335
I20250904 19:58:02.621472 12271 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.190:34335 every 8 connection(s)
I20250904 19:58:02.624883 12272 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:02.625030 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:02.630115 12274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:02.631649 12275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:02.631812 12272 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:02.632695 10338 server_base.cc:1047] running on GCE node
W20250904 19:58:02.633476 12277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:02.636229 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:02.636476 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:02.636675 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015882636656 us; error 0 us; skew 500 ppm
I20250904 19:58:02.637367 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:02.640177 10338 webserver.cc:480] Webserver started at http://127.10.24.189:36957/ using document root <none> and password file <none>
I20250904 19:58:02.640767 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:02.640988 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:02.641273 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:02.642338 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/master-1-root/instance:
uuid: "0c3ed212f5dc434f86d768969bd2402d"
format_stamp: "Formatted at 2025-09-04 19:58:02 on dist-test-slave-f3xj"
I20250904 19:58:02.645503 12272 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:02.647445 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
W20250904 19:58:02.650359 12272 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.189:36735: Network error: Client connection negotiation failed: client connection to 127.10.24.189:36735: connect: Connection refused (error 111)
I20250904 19:58:02.651085 12285 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:02.651875 10338 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20250904 19:58:02.652124 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/master-1-root
uuid: "0c3ed212f5dc434f86d768969bd2402d"
format_stamp: "Formatted at 2025-09-04 19:58:02 on dist-test-slave-f3xj"
I20250904 19:58:02.652343 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:02.667337 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:02.668494 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:02.703256 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.189:36735
I20250904 19:58:02.703409 12336 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.189:36735 every 8 connection(s)
I20250904 19:58:02.707008 12337 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:02.707086 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:02.708828 12272 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } attempt: 1
W20250904 19:58:02.713972 12339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:02.714555 12337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:58:02.715893 12341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:02.719524 12343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:02.721917 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:02.722985 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:02.723289 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:02.723496 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015882723474 us; error 0 us; skew 500 ppm
I20250904 19:58:02.724260 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:02.726323 12272 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:02.727535 10338 webserver.cc:480] Webserver started at http://127.10.24.188:34963/ using document root <none> and password file <none>
I20250904 19:58:02.728199 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:02.728473 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:02.728833 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:02.729840 12337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:02.730105 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/master-2-root/instance:
uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18"
format_stamp: "Formatted at 2025-09-04 19:58:02 on dist-test-slave-f3xj"
W20250904 19:58:02.731142 12272 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:34959: Network error: Client connection negotiation failed: client connection to 127.10.24.188:34959: connect: Connection refused (error 111)
I20250904 19:58:02.734750 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20250904 19:58:02.738372 12349 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:02.739145 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250904 19:58:02.739068 12337 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:02.739455 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/master-2-root
uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18"
format_stamp: "Formatted at 2025-09-04 19:58:02 on dist-test-slave-f3xj"
I20250904 19:58:02.739760 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20250904 19:58:02.743825 12337 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:34959: Network error: Client connection negotiation failed: client connection to 127.10.24.188:34959: connect: Connection refused (error 111)
I20250904 19:58:02.763511 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:02.764678 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:02.778381 12337 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } attempt: 1
W20250904 19:58:02.783051 12337 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:34959: Network error: Client connection negotiation failed: client connection to 127.10.24.188:34959: connect: Connection refused (error 111)
I20250904 19:58:02.793567 12272 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } attempt: 1
W20250904 19:58:02.797786 12272 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:34959: Network error: Client connection negotiation failed: client connection to 127.10.24.188:34959: connect: Connection refused (error 111)
I20250904 19:58:02.802846 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.188:34959
I20250904 19:58:02.802947 12401 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.188:34959 every 8 connection(s)
I20250904 19:58:02.805701 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 19:58:02.806607 12402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:02.812103 12402 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:02.821785 12402 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:02.829603 12402 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:02.843937 12402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18: Bootstrap starting.
I20250904 19:58:02.848218 12402 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:02.851899 12402 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18: No bootstrap required, opened a new log
I20250904 19:58:02.853838 12402 raft_consensus.cc:357] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "163c716beb0746a3a914158f4ad2b2de" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } } peers { permanent_uuid: "0c3ed212f5dc434f86d768969bd2402d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } } peers { permanent_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } }
I20250904 19:58:02.854359 12402 raft_consensus.cc:383] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:02.854579 12402 raft_consensus.cc:738] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f2bd66e1ea3406e96fc3e7c6a119a18, State: Initialized, Role: FOLLOWER
I20250904 19:58:02.855147 12402 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "163c716beb0746a3a914158f4ad2b2de" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } } peers { permanent_uuid: "0c3ed212f5dc434f86d768969bd2402d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } } peers { permanent_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } }
I20250904 19:58:02.856505 12337 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } attempt: 2
I20250904 19:58:02.856665 12407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "163c716beb0746a3a914158f4ad2b2de" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } } peers { permanent_uuid: "0c3ed212f5dc434f86d768969bd2402d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } } peers { permanent_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } } }
I20250904 19:58:02.857352 12407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:02.858023 12402 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:02.867228 12272 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } attempt: 2
I20250904 19:58:02.872733 12337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d: Bootstrap starting.
W20250904 19:58:02.875439 12418 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:58:02.875725 12418 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250904 19:58:02.878075 12337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:02.882004 12337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d: No bootstrap required, opened a new log
I20250904 19:58:02.882522 12272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de: Bootstrap starting.
I20250904 19:58:02.883889 12337 raft_consensus.cc:357] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "163c716beb0746a3a914158f4ad2b2de" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } } peers { permanent_uuid: "0c3ed212f5dc434f86d768969bd2402d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } } peers { permanent_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } }
I20250904 19:58:02.884382 12337 raft_consensus.cc:383] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:02.884585 12337 raft_consensus.cc:738] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c3ed212f5dc434f86d768969bd2402d, State: Initialized, Role: FOLLOWER
I20250904 19:58:02.885118 12337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "163c716beb0746a3a914158f4ad2b2de" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } } peers { permanent_uuid: "0c3ed212f5dc434f86d768969bd2402d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } } peers { permanent_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } }
I20250904 19:58:02.886749 12420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "163c716beb0746a3a914158f4ad2b2de" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } } peers { permanent_uuid: "0c3ed212f5dc434f86d768969bd2402d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } } peers { permanent_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } } }
I20250904 19:58:02.887193 12272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:02.887461 12420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:02.887852 12337 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:02.891713 12272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de: No bootstrap required, opened a new log
I20250904 19:58:02.893555 12272 raft_consensus.cc:357] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "163c716beb0746a3a914158f4ad2b2de" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } } peers { permanent_uuid: "0c3ed212f5dc434f86d768969bd2402d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } } peers { permanent_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } }
I20250904 19:58:02.894028 12272 raft_consensus.cc:383] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:02.894246 12272 raft_consensus.cc:738] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 163c716beb0746a3a914158f4ad2b2de, State: Initialized, Role: FOLLOWER
I20250904 19:58:02.894724 12272 consensus_queue.cc:260] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "163c716beb0746a3a914158f4ad2b2de" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } } peers { permanent_uuid: "0c3ed212f5dc434f86d768969bd2402d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } } peers { permanent_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } }
I20250904 19:58:02.896651 12428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "163c716beb0746a3a914158f4ad2b2de" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } } peers { permanent_uuid: "0c3ed212f5dc434f86d768969bd2402d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } } peers { permanent_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } } }
I20250904 19:58:02.897369 12428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:02.898089 12272 sys_catalog.cc:564] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de [sys.catalog]: configured and running, proceeding with master startup.
W20250904 19:58:02.902264 12433 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:58:02.902509 12433 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250904 19:58:02.908373 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20250904 19:58:02.908638 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20250904 19:58:02.910241 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:02.912007 12444 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:58:02.912317 12444 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20250904 19:58:02.916599 12445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:02.917671 12446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:02.918731 12448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:02.918928 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:02.919697 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:02.919878 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:02.920006 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015882919994 us; error 0 us; skew 500 ppm
I20250904 19:58:02.920451 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:02.922528 10338 webserver.cc:480] Webserver started at http://127.10.24.129:41083/ using document root <none> and password file <none>
I20250904 19:58:02.922937 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:02.923081 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:02.923270 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:02.924214 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/ts-0-root/instance:
uuid: "b74159907ac548c0b5c0681725ba713b"
format_stamp: "Formatted at 2025-09-04 19:58:02 on dist-test-slave-f3xj"
I20250904 19:58:02.927959 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20250904 19:58:02.930723 12453 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:02.931378 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250904 19:58:02.931646 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/ts-0-root
uuid: "b74159907ac548c0b5c0681725ba713b"
format_stamp: "Formatted at 2025-09-04 19:58:02 on dist-test-slave-f3xj"
I20250904 19:58:02.931881 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:02.950047 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:02.951033 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:02.952344 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:02.954502 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:02.954671 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:02.954900 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:02.955040 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:02.975652 12420 raft_consensus.cc:491] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:02.976222 12420 raft_consensus.cc:513] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "163c716beb0746a3a914158f4ad2b2de" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } } peers { permanent_uuid: "0c3ed212f5dc434f86d768969bd2402d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } } peers { permanent_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } }
I20250904 19:58:02.978374 12420 leader_election.cc:290] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 163c716beb0746a3a914158f4ad2b2de (127.10.24.190:34335), 3f2bd66e1ea3406e96fc3e7c6a119a18 (127.10.24.188:34959)
I20250904 19:58:02.979144 12247 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c3ed212f5dc434f86d768969bd2402d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "163c716beb0746a3a914158f4ad2b2de" is_pre_election: true
I20250904 19:58:02.979600 12377 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c3ed212f5dc434f86d768969bd2402d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" is_pre_election: true
I20250904 19:58:02.979841 12247 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c3ed212f5dc434f86d768969bd2402d in term 0.
I20250904 19:58:02.980249 12377 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c3ed212f5dc434f86d768969bd2402d in term 0.
I20250904 19:58:02.981047 12288 leader_election.cc:304] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c3ed212f5dc434f86d768969bd2402d, 163c716beb0746a3a914158f4ad2b2de; no voters:
I20250904 19:58:02.981884 12420 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:02.982218 12420 raft_consensus.cc:491] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:02.982465 12420 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:02.987005 12420 raft_consensus.cc:513] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "163c716beb0746a3a914158f4ad2b2de" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } } peers { permanent_uuid: "0c3ed212f5dc434f86d768969bd2402d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } } peers { permanent_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } }
I20250904 19:58:02.989166 12247 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c3ed212f5dc434f86d768969bd2402d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "163c716beb0746a3a914158f4ad2b2de"
I20250904 19:58:02.989503 12377 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0c3ed212f5dc434f86d768969bd2402d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18"
I20250904 19:58:02.989662 12420 leader_election.cc:290] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [CANDIDATE]: Term 1 election: Requested vote from peers 163c716beb0746a3a914158f4ad2b2de (127.10.24.190:34335), 3f2bd66e1ea3406e96fc3e7c6a119a18 (127.10.24.188:34959)
I20250904 19:58:02.989714 12247 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:02.990182 12377 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:02.996691 12377 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c3ed212f5dc434f86d768969bd2402d in term 1.
I20250904 19:58:02.996691 12247 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c3ed212f5dc434f86d768969bd2402d in term 1.
I20250904 19:58:02.997828 12287 leader_election.cc:304] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c3ed212f5dc434f86d768969bd2402d, 3f2bd66e1ea3406e96fc3e7c6a119a18; no voters:
I20250904 19:58:02.998623 12420 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:02.999924 12420 raft_consensus.cc:695] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [term 1 LEADER]: Becoming Leader. State: Replica: 0c3ed212f5dc434f86d768969bd2402d, State: Running, Role: LEADER
I20250904 19:58:03.000830 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.129:35261
I20250904 19:58:03.000994 12516 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.129:35261 every 8 connection(s)
I20250904 19:58:03.000726 12420 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "163c716beb0746a3a914158f4ad2b2de" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } } peers { permanent_uuid: "0c3ed212f5dc434f86d768969bd2402d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } } peers { permanent_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } }
I20250904 19:58:03.006026 12514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c3ed212f5dc434f86d768969bd2402d. Latest consensus state: current_term: 1 leader_uuid: "0c3ed212f5dc434f86d768969bd2402d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "163c716beb0746a3a914158f4ad2b2de" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } } peers { permanent_uuid: "0c3ed212f5dc434f86d768969bd2402d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } } peers { permanent_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } } }
I20250904 19:58:03.006740 12514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:03.008625 12521 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 19:58:03.015470 12521 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 19:58:03.025475 12517 heartbeater.cc:344] Connected to a master server at 127.10.24.189:36735
I20250904 19:58:03.025859 12517 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:03.026657 12517 heartbeater.cc:507] Master 127.10.24.189:36735 requested a full tablet report, sending...
I20250904 19:58:03.028815 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:03.028976 12302 ts_manager.cc:194] Registered new tserver with Master: b74159907ac548c0b5c0681725ba713b (127.10.24.129:35261)
I20250904 19:58:03.031543 12519 heartbeater.cc:344] Connected to a master server at 127.10.24.188:34959
I20250904 19:58:03.031998 12519 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:03.033414 12519 heartbeater.cc:507] Master 127.10.24.188:34959 requested a full tablet report, sending...
I20250904 19:58:03.034396 12522 heartbeater.cc:344] Connected to a master server at 127.10.24.190:34335
I20250904 19:58:03.034821 12522 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:03.035542 12522 heartbeater.cc:507] Master 127.10.24.190:34335 requested a full tablet report, sending...
I20250904 19:58:03.036320 12367 ts_manager.cc:194] Registered new tserver with Master: b74159907ac548c0b5c0681725ba713b (127.10.24.129:35261)
I20250904 19:58:03.038261 12237 ts_manager.cc:194] Registered new tserver with Master: b74159907ac548c0b5c0681725ba713b (127.10.24.129:35261)
W20250904 19:58:03.040620 12527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:03.041595 12528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:03.044147 12377 raft_consensus.cc:1273] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18 [term 1 FOLLOWER]: Refusing update from remote peer 0c3ed212f5dc434f86d768969bd2402d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:03.045679 12420 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:03.047183 12247 raft_consensus.cc:1273] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de [term 1 FOLLOWER]: Refusing update from remote peer 0c3ed212f5dc434f86d768969bd2402d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:03.049805 12420 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [LEADER]: Connected to new peer: Peer: permanent_uuid: "163c716beb0746a3a914158f4ad2b2de" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:03.052556 10338 server_base.cc:1047] running on GCE node
W20250904 19:58:03.053977 12531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:03.055047 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:03.055390 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:03.055677 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015883055657 us; error 0 us; skew 500 ppm
I20250904 19:58:03.056447 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:03.064532 12428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c3ed212f5dc434f86d768969bd2402d. Latest consensus state: current_term: 1 leader_uuid: "0c3ed212f5dc434f86d768969bd2402d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "163c716beb0746a3a914158f4ad2b2de" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } } peers { permanent_uuid: "0c3ed212f5dc434f86d768969bd2402d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } } peers { permanent_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } } }
I20250904 19:58:03.065380 12428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:03.066287 12407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c3ed212f5dc434f86d768969bd2402d. Latest consensus state: current_term: 1 leader_uuid: "0c3ed212f5dc434f86d768969bd2402d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "163c716beb0746a3a914158f4ad2b2de" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } } peers { permanent_uuid: "0c3ed212f5dc434f86d768969bd2402d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } } peers { permanent_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } } }
I20250904 19:58:03.067039 12407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:03.073995 10338 webserver.cc:480] Webserver started at http://127.10.24.130:37995/ using document root <none> and password file <none>
I20250904 19:58:03.073952 12514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c3ed212f5dc434f86d768969bd2402d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "163c716beb0746a3a914158f4ad2b2de" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } } peers { permanent_uuid: "0c3ed212f5dc434f86d768969bd2402d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } } peers { permanent_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } } }
I20250904 19:58:03.074874 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:03.074842 12514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:03.075232 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:03.075703 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:03.075665 12514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c3ed212f5dc434f86d768969bd2402d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "163c716beb0746a3a914158f4ad2b2de" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } } peers { permanent_uuid: "0c3ed212f5dc434f86d768969bd2402d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } } peers { permanent_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } } }
I20250904 19:58:03.076575 12514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:03.077343 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/ts-1-root/instance:
uuid: "fff38a0096c346a1a91d187e982e6a7c"
format_stamp: "Formatted at 2025-09-04 19:58:03 on dist-test-slave-f3xj"
I20250904 19:58:03.079109 12428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c3ed212f5dc434f86d768969bd2402d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "163c716beb0746a3a914158f4ad2b2de" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } } peers { permanent_uuid: "0c3ed212f5dc434f86d768969bd2402d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } } peers { permanent_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } } }
I20250904 19:58:03.079794 12428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:03.108189 12407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c3ed212f5dc434f86d768969bd2402d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "163c716beb0746a3a914158f4ad2b2de" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 34335 } } peers { permanent_uuid: "0c3ed212f5dc434f86d768969bd2402d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36735 } } peers { permanent_uuid: "3f2bd66e1ea3406e96fc3e7c6a119a18" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 34959 } } }
I20250904 19:58:03.108916 12407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:03.110186 12521 catalog_manager.cc:1349] Generated new cluster ID: 79887d944c4a4b8d937885b173e0cf23
I20250904 19:58:03.110487 12521 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 19:58:03.118177 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.040s user 0.004s sys 0.035s
I20250904 19:58:03.122123 12543 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:03.122911 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20250904 19:58:03.123183 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/ts-1-root
uuid: "fff38a0096c346a1a91d187e982e6a7c"
format_stamp: "Formatted at 2025-09-04 19:58:03 on dist-test-slave-f3xj"
I20250904 19:58:03.123426 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:03.144343 12521 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 19:58:03.146042 12521 catalog_manager.cc:1506] Loading token signing keys...
I20250904 19:58:03.153514 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:03.154973 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:03.156646 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:03.159720 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:03.159919 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:03.160141 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:03.160288 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:03.185276 12521 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d: Generated new TSK 0
I20250904 19:58:03.186249 12521 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 19:58:03.206360 12302 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47266
I20250904 19:58:03.219447 12517 heartbeater.cc:499] Master 127.10.24.189:36735 was elected leader, sending a full tablet report...
I20250904 19:58:03.232599 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.130:43635
I20250904 19:58:03.232682 12605 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.130:43635 every 8 connection(s)
I20250904 19:58:03.241568 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:03.255533 12613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:03.256867 12606 heartbeater.cc:344] Connected to a master server at 127.10.24.189:36735
I20250904 19:58:03.257308 12606 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:03.258239 12606 heartbeater.cc:507] Master 127.10.24.189:36735 requested a full tablet report, sending...
I20250904 19:58:03.260629 12302 ts_manager.cc:194] Registered new tserver with Master: fff38a0096c346a1a91d187e982e6a7c (127.10.24.130:43635)
I20250904 19:58:03.262318 12302 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47268
I20250904 19:58:03.269565 12607 heartbeater.cc:344] Connected to a master server at 127.10.24.188:34959
I20250904 19:58:03.269943 12607 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:03.270584 12607 heartbeater.cc:507] Master 127.10.24.188:34959 requested a full tablet report, sending...
I20250904 19:58:03.271384 12608 heartbeater.cc:344] Connected to a master server at 127.10.24.190:34335
W20250904 19:58:03.271616 12614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:03.271998 12608 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:03.273231 12608 heartbeater.cc:507] Master 127.10.24.190:34335 requested a full tablet report, sending...
I20250904 19:58:03.274240 12367 ts_manager.cc:194] Registered new tserver with Master: fff38a0096c346a1a91d187e982e6a7c (127.10.24.130:43635)
I20250904 19:58:03.275645 12237 ts_manager.cc:194] Registered new tserver with Master: fff38a0096c346a1a91d187e982e6a7c (127.10.24.130:43635)
I20250904 19:58:03.277484 12606 heartbeater.cc:499] Master 127.10.24.189:36735 was elected leader, sending a full tablet report...
I20250904 19:58:03.279783 10338 server_base.cc:1047] running on GCE node
W20250904 19:58:03.279913 12616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:03.280989 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:03.281216 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:03.281384 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015883281370 us; error 0 us; skew 500 ppm
I20250904 19:58:03.281919 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:03.284060 10338 webserver.cc:480] Webserver started at http://127.10.24.131:46211/ using document root <none> and password file <none>
I20250904 19:58:03.284509 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:03.284686 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:03.284925 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:03.285892 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/ts-2-root/instance:
uuid: "803bc66a2a8843729e84fb6cdf9b3f97"
format_stamp: "Formatted at 2025-09-04 19:58:03 on dist-test-slave-f3xj"
I20250904 19:58:03.292321 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20250904 19:58:03.295598 12621 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:03.296247 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250904 19:58:03.296559 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/ts-2-root
uuid: "803bc66a2a8843729e84fb6cdf9b3f97"
format_stamp: "Formatted at 2025-09-04 19:58:03 on dist-test-slave-f3xj"
I20250904 19:58:03.296825 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestTableConsistency.1757015869512910-10338-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:03.317205 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:03.318758 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:03.320642 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:03.322723 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:03.322925 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:03.323143 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:03.323297 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:03.381052 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.131:34249
I20250904 19:58:03.381161 12683 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.131:34249 every 8 connection(s)
I20250904 19:58:03.428864 12684 heartbeater.cc:344] Connected to a master server at 127.10.24.189:36735
I20250904 19:58:03.429337 12684 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:03.429586 12685 heartbeater.cc:344] Connected to a master server at 127.10.24.188:34959
I20250904 19:58:03.429977 12685 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:03.430567 12684 heartbeater.cc:507] Master 127.10.24.189:36735 requested a full tablet report, sending...
I20250904 19:58:03.432693 12302 ts_manager.cc:194] Registered new tserver with Master: 803bc66a2a8843729e84fb6cdf9b3f97 (127.10.24.131:34249)
I20250904 19:58:03.434352 12687 heartbeater.cc:344] Connected to a master server at 127.10.24.190:34335
I20250904 19:58:03.434391 12302 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47272
I20250904 19:58:03.434855 12687 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:03.435684 12687 heartbeater.cc:507] Master 127.10.24.190:34335 requested a full tablet report, sending...
I20250904 19:58:03.430717 12685 heartbeater.cc:507] Master 127.10.24.188:34959 requested a full tablet report, sending...
I20250904 19:58:03.438030 12236 ts_manager.cc:194] Registered new tserver with Master: 803bc66a2a8843729e84fb6cdf9b3f97 (127.10.24.131:34249)
I20250904 19:58:03.438687 12367 ts_manager.cc:194] Registered new tserver with Master: 803bc66a2a8843729e84fb6cdf9b3f97 (127.10.24.131:34249)
I20250904 19:58:03.439818 10338 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016472498s
I20250904 19:58:03.446928 12684 heartbeater.cc:499] Master 127.10.24.189:36735 was elected leader, sending a full tablet report...
I20250904 19:58:03.481338 12302 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47282:
name: "ksck-test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\n\000\000\000\004\001\024\000\000\000""\004\001\n\000\000\000\004\001\024\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 19:58:03.483269 12302 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table ksck-test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 19:58:03.581092 12570 tablet_service.cc:1468] Processing CreateTablet for tablet 5b6689934327466aa7ec1c997cf859e6 (DEFAULT_TABLE table=ksck-test-table [id=39900743dab34198986ab2fc890b1b37]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:03.581700 12571 tablet_service.cc:1468] Processing CreateTablet for tablet ec9e5c6d648d4d5bbe05b3adaf933da6 (DEFAULT_TABLE table=ksck-test-table [id=39900743dab34198986ab2fc890b1b37]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:03.582460 12570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b6689934327466aa7ec1c997cf859e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:03.584062 12569 tablet_service.cc:1468] Processing CreateTablet for tablet a987793569c94402868d6f76948e4573 (DEFAULT_TABLE table=ksck-test-table [id=39900743dab34198986ab2fc890b1b37]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:03.585299 12569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a987793569c94402868d6f76948e4573. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:03.586129 12481 tablet_service.cc:1468] Processing CreateTablet for tablet ec9e5c6d648d4d5bbe05b3adaf933da6 (DEFAULT_TABLE table=ksck-test-table [id=39900743dab34198986ab2fc890b1b37]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:03.587364 12481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec9e5c6d648d4d5bbe05b3adaf933da6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:03.589138 12571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec9e5c6d648d4d5bbe05b3adaf933da6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:03.592296 12649 tablet_service.cc:1468] Processing CreateTablet for tablet ec9e5c6d648d4d5bbe05b3adaf933da6 (DEFAULT_TABLE table=ksck-test-table [id=39900743dab34198986ab2fc890b1b37]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:03.593556 12649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec9e5c6d648d4d5bbe05b3adaf933da6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:03.596530 12648 tablet_service.cc:1468] Processing CreateTablet for tablet 5b6689934327466aa7ec1c997cf859e6 (DEFAULT_TABLE table=ksck-test-table [id=39900743dab34198986ab2fc890b1b37]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:03.597833 12648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b6689934327466aa7ec1c997cf859e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:03.600543 12647 tablet_service.cc:1468] Processing CreateTablet for tablet a987793569c94402868d6f76948e4573 (DEFAULT_TABLE table=ksck-test-table [id=39900743dab34198986ab2fc890b1b37]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:03.601784 12647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a987793569c94402868d6f76948e4573. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:03.622319 12706 tablet_bootstrap.cc:492] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97: Bootstrap starting.
I20250904 19:58:03.632611 12705 tablet_bootstrap.cc:492] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c: Bootstrap starting.
I20250904 19:58:03.637930 12480 tablet_service.cc:1468] Processing CreateTablet for tablet 5b6689934327466aa7ec1c997cf859e6 (DEFAULT_TABLE table=ksck-test-table [id=39900743dab34198986ab2fc890b1b37]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:03.639315 12480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5b6689934327466aa7ec1c997cf859e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:03.642244 12479 tablet_service.cc:1468] Processing CreateTablet for tablet a987793569c94402868d6f76948e4573 (DEFAULT_TABLE table=ksck-test-table [id=39900743dab34198986ab2fc890b1b37]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:03.646770 12706 tablet_bootstrap.cc:654] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:03.654083 12479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a987793569c94402868d6f76948e4573. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:03.664394 12706 tablet_bootstrap.cc:492] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97: No bootstrap required, opened a new log
I20250904 19:58:03.664729 12708 tablet_bootstrap.cc:492] T ec9e5c6d648d4d5bbe05b3adaf933da6 P b74159907ac548c0b5c0681725ba713b: Bootstrap starting.
I20250904 19:58:03.669420 12706 ts_tablet_manager.cc:1397] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97: Time spent bootstrapping tablet: real 0.047s user 0.005s sys 0.012s
I20250904 19:58:03.675724 12706 raft_consensus.cc:357] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:03.676635 12706 raft_consensus.cc:383] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:03.677018 12706 raft_consensus.cc:738] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 803bc66a2a8843729e84fb6cdf9b3f97, State: Initialized, Role: FOLLOWER
I20250904 19:58:03.677702 12708 tablet_bootstrap.cc:654] T ec9e5c6d648d4d5bbe05b3adaf933da6 P b74159907ac548c0b5c0681725ba713b: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:03.677857 12706 consensus_queue.cc:260] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:03.709549 12708 tablet_bootstrap.cc:492] T ec9e5c6d648d4d5bbe05b3adaf933da6 P b74159907ac548c0b5c0681725ba713b: No bootstrap required, opened a new log
I20250904 19:58:03.710248 12708 ts_tablet_manager.cc:1397] T ec9e5c6d648d4d5bbe05b3adaf933da6 P b74159907ac548c0b5c0681725ba713b: Time spent bootstrapping tablet: real 0.046s user 0.007s sys 0.007s
I20250904 19:58:03.716291 12705 tablet_bootstrap.cc:654] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:03.741447 12708 raft_consensus.cc:357] T ec9e5c6d648d4d5bbe05b3adaf933da6 P b74159907ac548c0b5c0681725ba713b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:03.742518 12708 raft_consensus.cc:383] T ec9e5c6d648d4d5bbe05b3adaf933da6 P b74159907ac548c0b5c0681725ba713b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:03.742897 12708 raft_consensus.cc:738] T ec9e5c6d648d4d5bbe05b3adaf933da6 P b74159907ac548c0b5c0681725ba713b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b74159907ac548c0b5c0681725ba713b, State: Initialized, Role: FOLLOWER
I20250904 19:58:03.743789 12708 consensus_queue.cc:260] T ec9e5c6d648d4d5bbe05b3adaf933da6 P b74159907ac548c0b5c0681725ba713b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:03.752300 12705 tablet_bootstrap.cc:492] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c: No bootstrap required, opened a new log
I20250904 19:58:03.755740 12705 ts_tablet_manager.cc:1397] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c: Time spent bootstrapping tablet: real 0.123s user 0.010s sys 0.000s
I20250904 19:58:03.758505 12705 raft_consensus.cc:357] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:03.761583 12706 ts_tablet_manager.cc:1428] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97: Time spent starting tablet: real 0.092s user 0.030s sys 0.053s
I20250904 19:58:03.763708 12705 raft_consensus.cc:383] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:03.764068 12705 raft_consensus.cc:738] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fff38a0096c346a1a91d187e982e6a7c, State: Initialized, Role: FOLLOWER
I20250904 19:58:03.764752 12706 tablet_bootstrap.cc:492] T 5b6689934327466aa7ec1c997cf859e6 P 803bc66a2a8843729e84fb6cdf9b3f97: Bootstrap starting.
I20250904 19:58:03.765041 12705 consensus_queue.cc:260] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:03.772886 12706 tablet_bootstrap.cc:654] T 5b6689934327466aa7ec1c997cf859e6 P 803bc66a2a8843729e84fb6cdf9b3f97: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:03.779762 12708 ts_tablet_manager.cc:1428] T ec9e5c6d648d4d5bbe05b3adaf933da6 P b74159907ac548c0b5c0681725ba713b: Time spent starting tablet: real 0.069s user 0.008s sys 0.002s
I20250904 19:58:03.786058 12706 tablet_bootstrap.cc:492] T 5b6689934327466aa7ec1c997cf859e6 P 803bc66a2a8843729e84fb6cdf9b3f97: No bootstrap required, opened a new log
I20250904 19:58:03.786563 12706 ts_tablet_manager.cc:1397] T 5b6689934327466aa7ec1c997cf859e6 P 803bc66a2a8843729e84fb6cdf9b3f97: Time spent bootstrapping tablet: real 0.022s user 0.014s sys 0.002s
I20250904 19:58:03.790447 12706 raft_consensus.cc:357] T 5b6689934327466aa7ec1c997cf859e6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:03.791297 12706 raft_consensus.cc:383] T 5b6689934327466aa7ec1c997cf859e6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:03.791684 12706 raft_consensus.cc:738] T 5b6689934327466aa7ec1c997cf859e6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 803bc66a2a8843729e84fb6cdf9b3f97, State: Initialized, Role: FOLLOWER
I20250904 19:58:03.792368 12706 consensus_queue.cc:260] T 5b6689934327466aa7ec1c997cf859e6 P 803bc66a2a8843729e84fb6cdf9b3f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:03.795073 12708 tablet_bootstrap.cc:492] T a987793569c94402868d6f76948e4573 P b74159907ac548c0b5c0681725ba713b: Bootstrap starting.
I20250904 19:58:03.797439 12706 ts_tablet_manager.cc:1428] T 5b6689934327466aa7ec1c997cf859e6 P 803bc66a2a8843729e84fb6cdf9b3f97: Time spent starting tablet: real 0.011s user 0.004s sys 0.004s
I20250904 19:58:03.801940 12706 tablet_bootstrap.cc:492] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97: Bootstrap starting.
I20250904 19:58:03.813350 12708 tablet_bootstrap.cc:654] T a987793569c94402868d6f76948e4573 P b74159907ac548c0b5c0681725ba713b: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:03.816905 12706 tablet_bootstrap.cc:654] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:03.819376 12705 ts_tablet_manager.cc:1428] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c: Time spent starting tablet: real 0.063s user 0.019s sys 0.019s
I20250904 19:58:03.824748 12705 tablet_bootstrap.cc:492] T ec9e5c6d648d4d5bbe05b3adaf933da6 P fff38a0096c346a1a91d187e982e6a7c: Bootstrap starting.
I20250904 19:58:03.839100 12708 tablet_bootstrap.cc:492] T a987793569c94402868d6f76948e4573 P b74159907ac548c0b5c0681725ba713b: No bootstrap required, opened a new log
I20250904 19:58:03.841308 12708 ts_tablet_manager.cc:1397] T a987793569c94402868d6f76948e4573 P b74159907ac548c0b5c0681725ba713b: Time spent bootstrapping tablet: real 0.046s user 0.011s sys 0.002s
I20250904 19:58:03.844249 12708 raft_consensus.cc:357] T a987793569c94402868d6f76948e4573 P b74159907ac548c0b5c0681725ba713b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:03.830775 12705 tablet_bootstrap.cc:654] T ec9e5c6d648d4d5bbe05b3adaf933da6 P fff38a0096c346a1a91d187e982e6a7c: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:03.845677 12708 raft_consensus.cc:383] T a987793569c94402868d6f76948e4573 P b74159907ac548c0b5c0681725ba713b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:03.846131 12708 raft_consensus.cc:738] T a987793569c94402868d6f76948e4573 P b74159907ac548c0b5c0681725ba713b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b74159907ac548c0b5c0681725ba713b, State: Initialized, Role: FOLLOWER
I20250904 19:58:03.846721 12708 consensus_queue.cc:260] T a987793569c94402868d6f76948e4573 P b74159907ac548c0b5c0681725ba713b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:03.848886 12706 tablet_bootstrap.cc:492] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97: No bootstrap required, opened a new log
I20250904 19:58:03.849383 12706 ts_tablet_manager.cc:1397] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97: Time spent bootstrapping tablet: real 0.048s user 0.010s sys 0.021s
I20250904 19:58:03.852854 12708 ts_tablet_manager.cc:1428] T a987793569c94402868d6f76948e4573 P b74159907ac548c0b5c0681725ba713b: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20250904 19:58:03.856576 12708 tablet_bootstrap.cc:492] T 5b6689934327466aa7ec1c997cf859e6 P b74159907ac548c0b5c0681725ba713b: Bootstrap starting.
I20250904 19:58:03.855199 12706 raft_consensus.cc:357] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:03.859876 12706 raft_consensus.cc:383] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:03.860248 12706 raft_consensus.cc:738] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 803bc66a2a8843729e84fb6cdf9b3f97, State: Initialized, Role: FOLLOWER
I20250904 19:58:03.860985 12706 consensus_queue.cc:260] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:03.862759 12706 ts_tablet_manager.cc:1428] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97: Time spent starting tablet: real 0.013s user 0.007s sys 0.000s
I20250904 19:58:03.863529 12705 tablet_bootstrap.cc:492] T ec9e5c6d648d4d5bbe05b3adaf933da6 P fff38a0096c346a1a91d187e982e6a7c: No bootstrap required, opened a new log
I20250904 19:58:03.864238 12705 ts_tablet_manager.cc:1397] T ec9e5c6d648d4d5bbe05b3adaf933da6 P fff38a0096c346a1a91d187e982e6a7c: Time spent bootstrapping tablet: real 0.040s user 0.010s sys 0.006s
I20250904 19:58:03.866873 12705 raft_consensus.cc:357] T ec9e5c6d648d4d5bbe05b3adaf933da6 P fff38a0096c346a1a91d187e982e6a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:03.867550 12708 tablet_bootstrap.cc:654] T 5b6689934327466aa7ec1c997cf859e6 P b74159907ac548c0b5c0681725ba713b: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:03.880522 12418 catalog_manager.cc:1261] Loaded cluster ID: 79887d944c4a4b8d937885b173e0cf23
I20250904 19:58:03.881592 12418 catalog_manager.cc:1554] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18: loading cluster ID for follower catalog manager: success
I20250904 19:58:03.867653 12705 raft_consensus.cc:383] T ec9e5c6d648d4d5bbe05b3adaf933da6 P fff38a0096c346a1a91d187e982e6a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:03.886991 12705 raft_consensus.cc:738] T ec9e5c6d648d4d5bbe05b3adaf933da6 P fff38a0096c346a1a91d187e982e6a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fff38a0096c346a1a91d187e982e6a7c, State: Initialized, Role: FOLLOWER
I20250904 19:58:03.887746 12705 consensus_queue.cc:260] T ec9e5c6d648d4d5bbe05b3adaf933da6 P fff38a0096c346a1a91d187e982e6a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:03.888792 12708 tablet_bootstrap.cc:492] T 5b6689934327466aa7ec1c997cf859e6 P b74159907ac548c0b5c0681725ba713b: No bootstrap required, opened a new log
I20250904 19:58:03.889374 12708 ts_tablet_manager.cc:1397] T 5b6689934327466aa7ec1c997cf859e6 P b74159907ac548c0b5c0681725ba713b: Time spent bootstrapping tablet: real 0.033s user 0.014s sys 0.012s
I20250904 19:58:03.890616 12418 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18: acquiring CA information for follower catalog manager: success
I20250904 19:58:03.891999 12708 raft_consensus.cc:357] T 5b6689934327466aa7ec1c997cf859e6 P b74159907ac548c0b5c0681725ba713b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:03.897197 12708 raft_consensus.cc:383] T 5b6689934327466aa7ec1c997cf859e6 P b74159907ac548c0b5c0681725ba713b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:03.897547 12708 raft_consensus.cc:738] T 5b6689934327466aa7ec1c997cf859e6 P b74159907ac548c0b5c0681725ba713b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b74159907ac548c0b5c0681725ba713b, State: Initialized, Role: FOLLOWER
I20250904 19:58:03.898152 12418 catalog_manager.cc:1604] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:58:03.898298 12708 consensus_queue.cc:260] T 5b6689934327466aa7ec1c997cf859e6 P b74159907ac548c0b5c0681725ba713b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:03.899117 12705 ts_tablet_manager.cc:1428] T ec9e5c6d648d4d5bbe05b3adaf933da6 P fff38a0096c346a1a91d187e982e6a7c: Time spent starting tablet: real 0.035s user 0.006s sys 0.001s
I20250904 19:58:03.901019 12708 ts_tablet_manager.cc:1428] T 5b6689934327466aa7ec1c997cf859e6 P b74159907ac548c0b5c0681725ba713b: Time spent starting tablet: real 0.011s user 0.003s sys 0.002s
I20250904 19:58:03.904013 12705 tablet_bootstrap.cc:492] T a987793569c94402868d6f76948e4573 P fff38a0096c346a1a91d187e982e6a7c: Bootstrap starting.
I20250904 19:58:03.910316 12705 tablet_bootstrap.cc:654] T a987793569c94402868d6f76948e4573 P fff38a0096c346a1a91d187e982e6a7c: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:03.917286 12444 catalog_manager.cc:1261] Loaded cluster ID: 79887d944c4a4b8d937885b173e0cf23
I20250904 19:58:03.917593 12444 catalog_manager.cc:1554] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de: loading cluster ID for follower catalog manager: success
I20250904 19:58:03.920116 12705 tablet_bootstrap.cc:492] T a987793569c94402868d6f76948e4573 P fff38a0096c346a1a91d187e982e6a7c: No bootstrap required, opened a new log
I20250904 19:58:03.920629 12705 ts_tablet_manager.cc:1397] T a987793569c94402868d6f76948e4573 P fff38a0096c346a1a91d187e982e6a7c: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.000s
I20250904 19:58:03.925101 12444 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de: acquiring CA information for follower catalog manager: success
I20250904 19:58:03.928407 12705 raft_consensus.cc:357] T a987793569c94402868d6f76948e4573 P fff38a0096c346a1a91d187e982e6a7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:03.929180 12705 raft_consensus.cc:383] T a987793569c94402868d6f76948e4573 P fff38a0096c346a1a91d187e982e6a7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:03.929471 12705 raft_consensus.cc:738] T a987793569c94402868d6f76948e4573 P fff38a0096c346a1a91d187e982e6a7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fff38a0096c346a1a91d187e982e6a7c, State: Initialized, Role: FOLLOWER
I20250904 19:58:03.930194 12705 consensus_queue.cc:260] T a987793569c94402868d6f76948e4573 P fff38a0096c346a1a91d187e982e6a7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:03.933478 12444 catalog_manager.cc:1604] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:58:03.935705 12705 ts_tablet_manager.cc:1428] T a987793569c94402868d6f76948e4573 P fff38a0096c346a1a91d187e982e6a7c: Time spent starting tablet: real 0.015s user 0.005s sys 0.001s
I20250904 19:58:03.959484 12709 raft_consensus.cc:491] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:03.959996 12709 raft_consensus.cc:513] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:03.962193 12709 leader_election.cc:290] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b74159907ac548c0b5c0681725ba713b (127.10.24.129:35261), fff38a0096c346a1a91d187e982e6a7c (127.10.24.130:43635)
I20250904 19:58:03.981863 12491 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "ec9e5c6d648d4d5bbe05b3adaf933da6" candidate_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b74159907ac548c0b5c0681725ba713b" is_pre_election: true
I20250904 19:58:03.982625 12491 raft_consensus.cc:2466] T ec9e5c6d648d4d5bbe05b3adaf933da6 P b74159907ac548c0b5c0681725ba713b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 803bc66a2a8843729e84fb6cdf9b3f97 in term 0.
I20250904 19:58:03.983763 12623 leader_election.cc:304] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 803bc66a2a8843729e84fb6cdf9b3f97, b74159907ac548c0b5c0681725ba713b; no voters:
I20250904 19:58:03.984547 12709 raft_consensus.cc:2802] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:03.984913 12709 raft_consensus.cc:491] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:03.985224 12709 raft_consensus.cc:3058] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:03.993002 12581 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "ec9e5c6d648d4d5bbe05b3adaf933da6" candidate_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fff38a0096c346a1a91d187e982e6a7c" is_pre_election: true
I20250904 19:58:03.993723 12581 raft_consensus.cc:2466] T ec9e5c6d648d4d5bbe05b3adaf933da6 P fff38a0096c346a1a91d187e982e6a7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 803bc66a2a8843729e84fb6cdf9b3f97 in term 0.
I20250904 19:58:03.996284 12709 raft_consensus.cc:513] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:03.998298 12709 leader_election.cc:290] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [CANDIDATE]: Term 1 election: Requested vote from peers b74159907ac548c0b5c0681725ba713b (127.10.24.129:35261), fff38a0096c346a1a91d187e982e6a7c (127.10.24.130:43635)
I20250904 19:58:03.998937 12491 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "ec9e5c6d648d4d5bbe05b3adaf933da6" candidate_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b74159907ac548c0b5c0681725ba713b"
I20250904 19:58:03.999132 12581 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "ec9e5c6d648d4d5bbe05b3adaf933da6" candidate_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fff38a0096c346a1a91d187e982e6a7c"
I20250904 19:58:03.999504 12491 raft_consensus.cc:3058] T ec9e5c6d648d4d5bbe05b3adaf933da6 P b74159907ac548c0b5c0681725ba713b [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:03.999765 12581 raft_consensus.cc:3058] T ec9e5c6d648d4d5bbe05b3adaf933da6 P fff38a0096c346a1a91d187e982e6a7c [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:04.005295 12581 raft_consensus.cc:2466] T ec9e5c6d648d4d5bbe05b3adaf933da6 P fff38a0096c346a1a91d187e982e6a7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 803bc66a2a8843729e84fb6cdf9b3f97 in term 1.
I20250904 19:58:04.006371 12625 leader_election.cc:304] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 803bc66a2a8843729e84fb6cdf9b3f97, fff38a0096c346a1a91d187e982e6a7c; no voters:
I20250904 19:58:04.015448 12709 raft_consensus.cc:2802] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:04.012732 12491 raft_consensus.cc:2466] T ec9e5c6d648d4d5bbe05b3adaf933da6 P b74159907ac548c0b5c0681725ba713b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 803bc66a2a8843729e84fb6cdf9b3f97 in term 1.
I20250904 19:58:04.017037 12709 raft_consensus.cc:695] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 1 LEADER]: Becoming Leader. State: Replica: 803bc66a2a8843729e84fb6cdf9b3f97, State: Running, Role: LEADER
I20250904 19:58:04.018837 12709 consensus_queue.cc:237] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:04.028937 12299 catalog_manager.cc:5582] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 reported cstate change: term changed from 0 to 1, leader changed from <none> to 803bc66a2a8843729e84fb6cdf9b3f97 (127.10.24.131). New cstate: current_term: 1 leader_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } health_report { overall_health: HEALTHY } } }
I20250904 19:58:04.099795 12713 raft_consensus.cc:491] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:04.100198 12713 raft_consensus.cc:513] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:04.101861 12709 raft_consensus.cc:491] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:04.102021 12713 leader_election.cc:290] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b74159907ac548c0b5c0681725ba713b (127.10.24.129:35261), 803bc66a2a8843729e84fb6cdf9b3f97 (127.10.24.131:34249)
I20250904 19:58:04.102383 12709 raft_consensus.cc:513] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:04.104276 12709 leader_election.cc:290] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b74159907ac548c0b5c0681725ba713b (127.10.24.129:35261), fff38a0096c346a1a91d187e982e6a7c (127.10.24.130:43635)
I20250904 19:58:04.108963 12491 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a987793569c94402868d6f76948e4573" candidate_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b74159907ac548c0b5c0681725ba713b" is_pre_election: true
I20250904 19:58:04.109606 12491 raft_consensus.cc:2466] T a987793569c94402868d6f76948e4573 P b74159907ac548c0b5c0681725ba713b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 803bc66a2a8843729e84fb6cdf9b3f97 in term 0.
I20250904 19:58:04.115083 12623 leader_election.cc:304] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 803bc66a2a8843729e84fb6cdf9b3f97, b74159907ac548c0b5c0681725ba713b; no voters:
I20250904 19:58:04.116695 12709 raft_consensus.cc:2802] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:04.117072 12709 raft_consensus.cc:491] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:04.117430 12709 raft_consensus.cc:3058] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:04.120957 12581 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a987793569c94402868d6f76948e4573" candidate_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fff38a0096c346a1a91d187e982e6a7c" is_pre_election: true
I20250904 19:58:04.121606 12581 raft_consensus.cc:2466] T a987793569c94402868d6f76948e4573 P fff38a0096c346a1a91d187e982e6a7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 803bc66a2a8843729e84fb6cdf9b3f97 in term 0.
I20250904 19:58:04.133407 12709 raft_consensus.cc:513] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:04.135113 12491 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5b6689934327466aa7ec1c997cf859e6" candidate_uuid: "fff38a0096c346a1a91d187e982e6a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b74159907ac548c0b5c0681725ba713b" is_pre_election: true
I20250904 19:58:04.136586 12491 raft_consensus.cc:2466] T 5b6689934327466aa7ec1c997cf859e6 P b74159907ac548c0b5c0681725ba713b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fff38a0096c346a1a91d187e982e6a7c in term 0.
I20250904 19:58:04.139492 12545 leader_election.cc:304] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b74159907ac548c0b5c0681725ba713b, fff38a0096c346a1a91d187e982e6a7c; no voters:
I20250904 19:58:04.146044 12713 raft_consensus.cc:2802] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:04.146430 12713 raft_consensus.cc:491] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:04.146817 12713 raft_consensus.cc:3058] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:04.147944 12709 leader_election.cc:290] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [CANDIDATE]: Term 1 election: Requested vote from peers b74159907ac548c0b5c0681725ba713b (127.10.24.129:35261), fff38a0096c346a1a91d187e982e6a7c (127.10.24.130:43635)
I20250904 19:58:04.148705 12659 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5b6689934327466aa7ec1c997cf859e6" candidate_uuid: "fff38a0096c346a1a91d187e982e6a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" is_pre_election: true
I20250904 19:58:04.149370 12659 raft_consensus.cc:2466] T 5b6689934327466aa7ec1c997cf859e6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fff38a0096c346a1a91d187e982e6a7c in term 0.
I20250904 19:58:04.149811 12581 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a987793569c94402868d6f76948e4573" candidate_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fff38a0096c346a1a91d187e982e6a7c"
I20250904 19:58:04.150398 12581 raft_consensus.cc:3058] T a987793569c94402868d6f76948e4573 P fff38a0096c346a1a91d187e982e6a7c [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:04.150700 12491 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a987793569c94402868d6f76948e4573" candidate_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b74159907ac548c0b5c0681725ba713b"
I20250904 19:58:04.151268 12491 raft_consensus.cc:3058] T a987793569c94402868d6f76948e4573 P b74159907ac548c0b5c0681725ba713b [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:04.155294 12713 raft_consensus.cc:513] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:04.157855 12491 raft_consensus.cc:2466] T a987793569c94402868d6f76948e4573 P b74159907ac548c0b5c0681725ba713b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 803bc66a2a8843729e84fb6cdf9b3f97 in term 1.
I20250904 19:58:04.159304 12623 leader_election.cc:304] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 803bc66a2a8843729e84fb6cdf9b3f97, b74159907ac548c0b5c0681725ba713b; no voters:
I20250904 19:58:04.159178 12659 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5b6689934327466aa7ec1c997cf859e6" candidate_uuid: "fff38a0096c346a1a91d187e982e6a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "803bc66a2a8843729e84fb6cdf9b3f97"
I20250904 19:58:04.160995 12659 raft_consensus.cc:3058] T 5b6689934327466aa7ec1c997cf859e6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:04.164877 12709 raft_consensus.cc:2802] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:04.167140 12491 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5b6689934327466aa7ec1c997cf859e6" candidate_uuid: "fff38a0096c346a1a91d187e982e6a7c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b74159907ac548c0b5c0681725ba713b"
I20250904 19:58:04.169268 12491 raft_consensus.cc:3058] T 5b6689934327466aa7ec1c997cf859e6 P b74159907ac548c0b5c0681725ba713b [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:04.170885 12659 raft_consensus.cc:2466] T 5b6689934327466aa7ec1c997cf859e6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fff38a0096c346a1a91d187e982e6a7c in term 1.
I20250904 19:58:04.167337 12581 raft_consensus.cc:2466] T a987793569c94402868d6f76948e4573 P fff38a0096c346a1a91d187e982e6a7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 803bc66a2a8843729e84fb6cdf9b3f97 in term 1.
I20250904 19:58:04.171998 12547 leader_election.cc:304] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 803bc66a2a8843729e84fb6cdf9b3f97, fff38a0096c346a1a91d187e982e6a7c; no voters:
I20250904 19:58:04.172470 12713 leader_election.cc:290] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [CANDIDATE]: Term 1 election: Requested vote from peers b74159907ac548c0b5c0681725ba713b (127.10.24.129:35261), 803bc66a2a8843729e84fb6cdf9b3f97 (127.10.24.131:34249)
I20250904 19:58:04.172123 12709 raft_consensus.cc:695] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 1 LEADER]: Becoming Leader. State: Replica: 803bc66a2a8843729e84fb6cdf9b3f97, State: Running, Role: LEADER
I20250904 19:58:04.173493 12709 consensus_queue.cc:237] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:04.174492 12713 raft_consensus.cc:2802] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:04.178814 12491 raft_consensus.cc:2466] T 5b6689934327466aa7ec1c997cf859e6 P b74159907ac548c0b5c0681725ba713b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fff38a0096c346a1a91d187e982e6a7c in term 1.
I20250904 19:58:04.182168 12713 raft_consensus.cc:695] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [term 1 LEADER]: Becoming Leader. State: Replica: fff38a0096c346a1a91d187e982e6a7c, State: Running, Role: LEADER
I20250904 19:58:04.186035 12713 consensus_queue.cc:237] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } }
I20250904 19:58:04.188244 12300 catalog_manager.cc:5582] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 reported cstate change: term changed from 0 to 1, leader changed from <none> to 803bc66a2a8843729e84fb6cdf9b3f97 (127.10.24.131). New cstate: current_term: 1 leader_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } health_report { overall_health: HEALTHY } } }
I20250904 19:58:04.196015 12302 catalog_manager.cc:5582] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c reported cstate change: term changed from 0 to 1, leader changed from <none> to fff38a0096c346a1a91d187e982e6a7c (127.10.24.130). New cstate: current_term: 1 leader_uuid: "fff38a0096c346a1a91d187e982e6a7c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 } health_report { overall_health: UNKNOWN } } }
I20250904 19:58:04.553709 12709 consensus_queue.cc:1035] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:04.584700 12713 consensus_queue.cc:1035] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:04.586607 12709 consensus_queue.cc:1035] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fff38a0096c346a1a91d187e982e6a7c" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:04.632508 12709 consensus_queue.cc:1035] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250904 19:58:04.639017 12733 consensus_queue.cc:1035] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [LEADER]: Connected to new peer: Peer: permanent_uuid: "803bc66a2a8843729e84fb6cdf9b3f97" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 34249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:04.730862 12709 consensus_queue.cc:1035] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b74159907ac548c0b5c0681725ba713b" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 35261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:05.056878 12480 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250904 19:58:05.061866 12649 tablet_service.cc:1430] Tablet server has 2 leaders and 0 scanners
I20250904 19:58:05.088156 12569 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250904 19:58:05.447234 10338 tablet_server.cc:178] TabletServer@127.10.24.129:0 shutting down...
I20250904 19:58:05.493638 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:05.494297 10338 tablet_replica.cc:331] T 5b6689934327466aa7ec1c997cf859e6 P b74159907ac548c0b5c0681725ba713b: stopping tablet replica
I20250904 19:58:05.495126 10338 raft_consensus.cc:2241] T 5b6689934327466aa7ec1c997cf859e6 P b74159907ac548c0b5c0681725ba713b [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:05.495646 10338 raft_consensus.cc:2270] T 5b6689934327466aa7ec1c997cf859e6 P b74159907ac548c0b5c0681725ba713b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:05.497629 10338 tablet_replica.cc:331] T a987793569c94402868d6f76948e4573 P b74159907ac548c0b5c0681725ba713b: stopping tablet replica
I20250904 19:58:05.498113 10338 raft_consensus.cc:2241] T a987793569c94402868d6f76948e4573 P b74159907ac548c0b5c0681725ba713b [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:05.498513 10338 raft_consensus.cc:2270] T a987793569c94402868d6f76948e4573 P b74159907ac548c0b5c0681725ba713b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:05.504036 10338 tablet_replica.cc:331] T ec9e5c6d648d4d5bbe05b3adaf933da6 P b74159907ac548c0b5c0681725ba713b: stopping tablet replica
I20250904 19:58:05.504490 10338 raft_consensus.cc:2241] T ec9e5c6d648d4d5bbe05b3adaf933da6 P b74159907ac548c0b5c0681725ba713b [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:05.504935 10338 raft_consensus.cc:2270] T ec9e5c6d648d4d5bbe05b3adaf933da6 P b74159907ac548c0b5c0681725ba713b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:05.517815 10338 tablet_server.cc:195] TabletServer@127.10.24.129:0 shutdown complete.
I20250904 19:58:05.532294 10338 tablet_server.cc:178] TabletServer@127.10.24.130:0 shutting down...
W20250904 19:58:05.553517 12625 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 26 similar messages]
W20250904 19:58:05.560890 12625 consensus_peers.cc:489] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 -> Peer fff38a0096c346a1a91d187e982e6a7c (127.10.24.130:43635): Couldn't send request to peer fff38a0096c346a1a91d187e982e6a7c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250904 19:58:05.569743 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:05.570253 10338 tablet_replica.cc:331] T a987793569c94402868d6f76948e4573 P fff38a0096c346a1a91d187e982e6a7c: stopping tablet replica
I20250904 19:58:05.570752 10338 raft_consensus.cc:2241] T a987793569c94402868d6f76948e4573 P fff38a0096c346a1a91d187e982e6a7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:05.571139 10338 raft_consensus.cc:2270] T a987793569c94402868d6f76948e4573 P fff38a0096c346a1a91d187e982e6a7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:05.573374 10338 tablet_replica.cc:331] T ec9e5c6d648d4d5bbe05b3adaf933da6 P fff38a0096c346a1a91d187e982e6a7c: stopping tablet replica
I20250904 19:58:05.573937 10338 raft_consensus.cc:2241] T ec9e5c6d648d4d5bbe05b3adaf933da6 P fff38a0096c346a1a91d187e982e6a7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:05.574317 10338 raft_consensus.cc:2270] T ec9e5c6d648d4d5bbe05b3adaf933da6 P fff38a0096c346a1a91d187e982e6a7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:05.576354 10338 tablet_replica.cc:331] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c: stopping tablet replica
I20250904 19:58:05.576804 10338 raft_consensus.cc:2241] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:05.577672 10338 raft_consensus.cc:2270] T 5b6689934327466aa7ec1c997cf859e6 P fff38a0096c346a1a91d187e982e6a7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:05.587620 10338 tablet_server.cc:195] TabletServer@127.10.24.130:0 shutdown complete.
I20250904 19:58:05.600505 10338 tablet_server.cc:178] TabletServer@127.10.24.131:0 shutting down...
W20250904 19:58:05.612037 12623 consensus_peers.cc:489] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 -> Peer b74159907ac548c0b5c0681725ba713b (127.10.24.129:35261): Couldn't send request to peer b74159907ac548c0b5c0681725ba713b. Status: Network error: Client connection negotiation failed: client connection to 127.10.24.129:35261: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 19:58:05.621819 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:05.622326 10338 tablet_replica.cc:331] T 5b6689934327466aa7ec1c997cf859e6 P 803bc66a2a8843729e84fb6cdf9b3f97: stopping tablet replica
I20250904 19:58:05.622881 10338 raft_consensus.cc:2241] T 5b6689934327466aa7ec1c997cf859e6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:05.623256 10338 raft_consensus.cc:2270] T 5b6689934327466aa7ec1c997cf859e6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:05.625276 10338 tablet_replica.cc:331] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97: stopping tablet replica
I20250904 19:58:05.625823 10338 raft_consensus.cc:2241] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:05.626582 10338 raft_consensus.cc:2270] T ec9e5c6d648d4d5bbe05b3adaf933da6 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:05.629046 10338 tablet_replica.cc:331] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97: stopping tablet replica
I20250904 19:58:05.629462 10338 raft_consensus.cc:2241] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:05.630045 10338 raft_consensus.cc:2270] T a987793569c94402868d6f76948e4573 P 803bc66a2a8843729e84fb6cdf9b3f97 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:05.649585 10338 tablet_server.cc:195] TabletServer@127.10.24.131:0 shutdown complete.
I20250904 19:58:05.659108 10338 master.cc:561] Master@127.10.24.190:34335 shutting down...
I20250904 19:58:05.673386 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:05.673877 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:05.674145 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 163c716beb0746a3a914158f4ad2b2de: stopping tablet replica
I20250904 19:58:05.692334 10338 master.cc:583] Master@127.10.24.190:34335 shutdown complete.
I20250904 19:58:05.702621 10338 master.cc:561] Master@127.10.24.189:36735 shutting down...
I20250904 19:58:05.716714 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:05.717610 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:05.717932 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 0c3ed212f5dc434f86d768969bd2402d: stopping tablet replica
I20250904 19:58:05.737102 10338 master.cc:583] Master@127.10.24.189:36735 shutdown complete.
I20250904 19:58:05.748606 10338 master.cc:561] Master@127.10.24.188:34959 shutting down...
I20250904 19:58:05.763064 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:05.763547 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:05.763845 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 3f2bd66e1ea3406e96fc3e7c6a119a18: stopping tablet replica
I20250904 19:58:05.781387 10338 master.cc:583] Master@127.10.24.188:34959 shutdown complete.
[ OK ] RemoteKsckTest.TestTableConsistency (3266 ms)
[ RUN ] RemoteKsckTest.TestChecksum
I20250904 19:58:05.809585 10338 test_util.cc:276] Using random seed: 599590811
I20250904 19:58:05.810776 10338 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.24.190:35721,127.10.24.189:36539,127.10.24.188:38349
I20250904 19:58:05.812497 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:05.817178 12794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:05.817255 12795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:05.824752 12797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:05.825193 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:05.825887 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:05.826076 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:05.826211 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015885826194 us; error 0 us; skew 500 ppm
I20250904 19:58:05.826685 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:05.828758 10338 webserver.cc:480] Webserver started at http://127.10.24.190:39545/ using document root <none> and password file <none>
I20250904 19:58:05.829216 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:05.829382 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:05.829612 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:05.830562 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/master-0-root/instance:
uuid: "3f68358e98174dbe96102699d238b1fe"
format_stamp: "Formatted at 2025-09-04 19:58:05 on dist-test-slave-f3xj"
I20250904 19:58:05.834501 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20250904 19:58:05.837432 12802 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:05.838155 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20250904 19:58:05.838414 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/master-0-root
uuid: "3f68358e98174dbe96102699d238b1fe"
format_stamp: "Formatted at 2025-09-04 19:58:05 on dist-test-slave-f3xj"
I20250904 19:58:05.838639 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:05.857574 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:05.858471 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:05.890362 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.190:35721
I20250904 19:58:05.890460 12853 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.190:35721 every 8 connection(s)
I20250904 19:58:05.893869 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:05.893982 12854 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20250904 19:58:05.898733 12856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:05.900514 12857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:05.901481 12854 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:58:05.901660 12859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:05.902074 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:05.902879 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:05.903088 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:05.903247 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015885903233 us; error 0 us; skew 500 ppm
I20250904 19:58:05.903951 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:05.906494 10338 webserver.cc:480] Webserver started at http://127.10.24.189:45535/ using document root <none> and password file <none>
I20250904 19:58:05.907047 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:05.907227 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:05.907451 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:05.908480 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/master-1-root/instance:
uuid: "fe26647b97c74957a7d3a27c19b6fe0d"
format_stamp: "Formatted at 2025-09-04 19:58:05 on dist-test-slave-f3xj"
I20250904 19:58:05.913455 12854 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:05.913789 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20250904 19:58:05.917918 12867 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
W20250904 19:58:05.918428 12854 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.189:36539: Network error: Client connection negotiation failed: client connection to 127.10.24.189:36539: connect: Connection refused (error 111)
I20250904 19:58:05.918706 10338 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20250904 19:58:05.918960 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/master-1-root
uuid: "fe26647b97c74957a7d3a27c19b6fe0d"
format_stamp: "Formatted at 2025-09-04 19:58:05 on dist-test-slave-f3xj"
I20250904 19:58:05.919183 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:05.945474 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:05.946435 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:05.948933 12854 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } attempt: 1
W20250904 19:58:05.953270 12854 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.189:36539: Network error: Client connection negotiation failed: client connection to 127.10.24.189:36539: connect: Connection refused (error 111)
I20250904 19:58:05.980985 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.189:36539
I20250904 19:58:05.981082 12918 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.189:36539 every 8 connection(s)
I20250904 19:58:05.984515 12919 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:05.984678 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:05.989516 12921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:05.991479 12922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:05.991715 12919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:58:05.992607 12924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:05.992947 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:05.994002 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:05.994199 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:05.994354 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015885994335 us; error 0 us; skew 500 ppm
I20250904 19:58:05.994807 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:05.996672 12854 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } attempt: 2
I20250904 19:58:05.999909 10338 webserver.cc:480] Webserver started at http://127.10.24.188:46791/ using document root <none> and password file <none>
I20250904 19:58:06.000584 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:06.000847 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:06.001202 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:06.002570 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/master-2-root/instance:
uuid: "5b38e3ee13f54866a19b1be0ff824b83"
format_stamp: "Formatted at 2025-09-04 19:58:05 on dist-test-slave-f3xj"
I20250904 19:58:06.004148 12919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:06.008263 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20250904 19:58:06.012272 12932 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:06.013247 10338 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20250904 19:58:06.013571 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/master-2-root
uuid: "5b38e3ee13f54866a19b1be0ff824b83"
format_stamp: "Formatted at 2025-09-04 19:58:05 on dist-test-slave-f3xj"
I20250904 19:58:06.013891 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:06.017474 12854 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:06.019709 12919 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:58:06.023191 12854 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:38349: Network error: Client connection negotiation failed: client connection to 127.10.24.188:38349: connect: Connection refused (error 111)
W20250904 19:58:06.025295 12919 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:38349: Network error: Client connection negotiation failed: client connection to 127.10.24.188:38349: connect: Connection refused (error 111)
I20250904 19:58:06.036387 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:06.037369 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:06.055661 12854 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } attempt: 1
W20250904 19:58:06.060577 12854 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:38349: Network error: Client connection negotiation failed: client connection to 127.10.24.188:38349: connect: Connection refused (error 111)
I20250904 19:58:06.074378 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.188:38349
I20250904 19:58:06.074482 12984 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.188:38349 every 8 connection(s)
I20250904 19:58:06.077817 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 19:58:06.078697 12985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:06.078713 12919 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } attempt: 1
I20250904 19:58:06.086313 12985 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:06.094305 12919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d: Bootstrap starting.
I20250904 19:58:06.096470 12985 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:06.099339 12919 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:06.103786 12919 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d: No bootstrap required, opened a new log
I20250904 19:58:06.104132 12854 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } attempt: 2
I20250904 19:58:06.105903 12985 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:06.106287 12919 raft_consensus.cc:357] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f68358e98174dbe96102699d238b1fe" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } } peers { permanent_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } } peers { permanent_uuid: "5b38e3ee13f54866a19b1be0ff824b83" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } }
I20250904 19:58:06.106891 12919 raft_consensus.cc:383] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:06.107147 12919 raft_consensus.cc:738] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe26647b97c74957a7d3a27c19b6fe0d, State: Initialized, Role: FOLLOWER
I20250904 19:58:06.107738 12919 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f68358e98174dbe96102699d238b1fe" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } } peers { permanent_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } } peers { permanent_uuid: "5b38e3ee13f54866a19b1be0ff824b83" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } }
I20250904 19:58:06.110216 12991 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f68358e98174dbe96102699d238b1fe" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } } peers { permanent_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } } peers { permanent_uuid: "5b38e3ee13f54866a19b1be0ff824b83" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } } }
I20250904 19:58:06.110975 12991 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:06.111848 12919 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:06.122175 12854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe: Bootstrap starting.
W20250904 19:58:06.127995 13002 catalog_manager.cc:1560] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20250904 19:58:06.128013 12854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe: Neither blocks nor log segments found. Creating new log.
W20250904 19:58:06.128398 13002 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250904 19:58:06.131217 12985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83: Bootstrap starting.
I20250904 19:58:06.132633 12854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe: No bootstrap required, opened a new log
I20250904 19:58:06.135257 12854 raft_consensus.cc:357] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f68358e98174dbe96102699d238b1fe" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } } peers { permanent_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } } peers { permanent_uuid: "5b38e3ee13f54866a19b1be0ff824b83" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } }
I20250904 19:58:06.135988 12854 raft_consensus.cc:383] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:06.136276 12854 raft_consensus.cc:738] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f68358e98174dbe96102699d238b1fe, State: Initialized, Role: FOLLOWER
I20250904 19:58:06.137063 12985 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:06.136963 12854 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f68358e98174dbe96102699d238b1fe" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } } peers { permanent_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } } peers { permanent_uuid: "5b38e3ee13f54866a19b1be0ff824b83" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } }
I20250904 19:58:06.138785 13004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f68358e98174dbe96102699d238b1fe" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } } peers { permanent_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } } peers { permanent_uuid: "5b38e3ee13f54866a19b1be0ff824b83" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } } }
I20250904 19:58:06.139298 13004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:06.140442 12854 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:06.141801 12985 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83: No bootstrap required, opened a new log
I20250904 19:58:06.144366 12985 raft_consensus.cc:357] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f68358e98174dbe96102699d238b1fe" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } } peers { permanent_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } } peers { permanent_uuid: "5b38e3ee13f54866a19b1be0ff824b83" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } }
I20250904 19:58:06.145051 12985 raft_consensus.cc:383] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:06.145337 12985 raft_consensus.cc:738] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b38e3ee13f54866a19b1be0ff824b83, State: Initialized, Role: FOLLOWER
I20250904 19:58:06.146045 12985 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f68358e98174dbe96102699d238b1fe" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } } peers { permanent_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } } peers { permanent_uuid: "5b38e3ee13f54866a19b1be0ff824b83" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } }
I20250904 19:58:06.147786 13011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f68358e98174dbe96102699d238b1fe" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } } peers { permanent_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } } peers { permanent_uuid: "5b38e3ee13f54866a19b1be0ff824b83" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } } }
I20250904 19:58:06.148303 13011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:06.149596 12985 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:06.152559 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20250904 19:58:06.152827 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
W20250904 19:58:06.155673 13017 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:58:06.155947 13017 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250904 19:58:06.162134 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:06.163668 13028 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:58:06.163936 13028 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20250904 19:58:06.167550 13029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:06.168679 13030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:06.170148 13032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:06.170264 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:06.171077 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:06.171273 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:06.171422 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015886171406 us; error 0 us; skew 500 ppm
I20250904 19:58:06.171907 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:06.173931 10338 webserver.cc:480] Webserver started at http://127.10.24.129:35641/ using document root <none> and password file <none>
I20250904 19:58:06.174379 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:06.174551 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:06.174777 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:06.175817 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/ts-0-root/instance:
uuid: "10f860a8c34748d09ee747a660374f2a"
format_stamp: "Formatted at 2025-09-04 19:58:06 on dist-test-slave-f3xj"
I20250904 19:58:06.179661 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20250904 19:58:06.182538 13037 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:06.183177 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250904 19:58:06.183435 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/ts-0-root
uuid: "10f860a8c34748d09ee747a660374f2a"
format_stamp: "Formatted at 2025-09-04 19:58:06 on dist-test-slave-f3xj"
I20250904 19:58:06.183723 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:06.193560 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:06.194480 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:06.195755 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:06.197782 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:06.197984 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:06.198191 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:06.198334 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:06.211386 13004 raft_consensus.cc:491] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:06.211880 13004 raft_consensus.cc:513] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f68358e98174dbe96102699d238b1fe" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } } peers { permanent_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } } peers { permanent_uuid: "5b38e3ee13f54866a19b1be0ff824b83" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } }
I20250904 19:58:06.213631 13004 leader_election.cc:290] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe26647b97c74957a7d3a27c19b6fe0d (127.10.24.189:36539), 5b38e3ee13f54866a19b1be0ff824b83 (127.10.24.188:38349)
I20250904 19:58:06.214576 12894 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f68358e98174dbe96102699d238b1fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" is_pre_election: true
I20250904 19:58:06.214802 12960 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f68358e98174dbe96102699d238b1fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b38e3ee13f54866a19b1be0ff824b83" is_pre_election: true
I20250904 19:58:06.215318 12894 raft_consensus.cc:2466] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f68358e98174dbe96102699d238b1fe in term 0.
I20250904 19:58:06.215437 12960 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f68358e98174dbe96102699d238b1fe in term 0.
I20250904 19:58:06.216674 12804 leader_election.cc:304] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f68358e98174dbe96102699d238b1fe, fe26647b97c74957a7d3a27c19b6fe0d; no voters:
I20250904 19:58:06.217499 13004 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:06.217887 13004 raft_consensus.cc:491] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:06.218202 13004 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:06.224467 13004 raft_consensus.cc:513] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f68358e98174dbe96102699d238b1fe" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } } peers { permanent_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } } peers { permanent_uuid: "5b38e3ee13f54866a19b1be0ff824b83" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } }
I20250904 19:58:06.226274 13004 leader_election.cc:290] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [CANDIDATE]: Term 1 election: Requested vote from peers fe26647b97c74957a7d3a27c19b6fe0d (127.10.24.189:36539), 5b38e3ee13f54866a19b1be0ff824b83 (127.10.24.188:38349)
I20250904 19:58:06.226714 12894 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f68358e98174dbe96102699d238b1fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe26647b97c74957a7d3a27c19b6fe0d"
I20250904 19:58:06.226922 12960 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f68358e98174dbe96102699d238b1fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b38e3ee13f54866a19b1be0ff824b83"
I20250904 19:58:06.227355 12894 raft_consensus.cc:3058] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:06.227622 12960 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:06.234205 12960 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f68358e98174dbe96102699d238b1fe in term 1.
I20250904 19:58:06.234205 12894 raft_consensus.cc:2466] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f68358e98174dbe96102699d238b1fe in term 1.
I20250904 19:58:06.235513 12804 leader_election.cc:304] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f68358e98174dbe96102699d238b1fe, fe26647b97c74957a7d3a27c19b6fe0d; no voters:
I20250904 19:58:06.236474 13004 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:06.237592 13004 raft_consensus.cc:695] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [term 1 LEADER]: Becoming Leader. State: Replica: 3f68358e98174dbe96102699d238b1fe, State: Running, Role: LEADER
I20250904 19:58:06.238338 13004 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f68358e98174dbe96102699d238b1fe" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } } peers { permanent_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } } peers { permanent_uuid: "5b38e3ee13f54866a19b1be0ff824b83" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } }
I20250904 19:58:06.243386 13092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f68358e98174dbe96102699d238b1fe. Latest consensus state: current_term: 1 leader_uuid: "3f68358e98174dbe96102699d238b1fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f68358e98174dbe96102699d238b1fe" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } } peers { permanent_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } } peers { permanent_uuid: "5b38e3ee13f54866a19b1be0ff824b83" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } } }
I20250904 19:58:06.243840 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.129:44773
I20250904 19:58:06.243942 13101 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.129:44773 every 8 connection(s)
I20250904 19:58:06.244052 13092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:06.245813 13102 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 19:58:06.253115 13102 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 19:58:06.260541 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:06.264189 12960 raft_consensus.cc:1273] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83 [term 1 FOLLOWER]: Refusing update from remote peer 3f68358e98174dbe96102699d238b1fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:06.266814 13004 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b38e3ee13f54866a19b1be0ff824b83" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:06.269268 12894 raft_consensus.cc:1273] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d [term 1 FOLLOWER]: Refusing update from remote peer 3f68358e98174dbe96102699d238b1fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:06.269994 13103 heartbeater.cc:344] Connected to a master server at 127.10.24.188:38349
I20250904 19:58:06.270449 13103 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:06.270995 13092 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:06.271332 13103 heartbeater.cc:507] Master 127.10.24.188:38349 requested a full tablet report, sending...
I20250904 19:58:06.274194 12950 ts_manager.cc:194] Registered new tserver with Master: 10f860a8c34748d09ee747a660374f2a (127.10.24.129:44773)
W20250904 19:58:06.274801 13111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:06.280715 13112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:06.284225 13104 heartbeater.cc:344] Connected to a master server at 127.10.24.189:36539
I20250904 19:58:06.284668 13104 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:06.285658 13104 heartbeater.cc:507] Master 127.10.24.189:36539 requested a full tablet report, sending...
W20250904 19:58:06.289469 13116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:06.290396 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:06.291826 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:06.292162 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:06.292372 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015886292354 us; error 0 us; skew 500 ppm
I20250904 19:58:06.293044 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:06.295877 12991 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f68358e98174dbe96102699d238b1fe. Latest consensus state: current_term: 1 leader_uuid: "3f68358e98174dbe96102699d238b1fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f68358e98174dbe96102699d238b1fe" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } } peers { permanent_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } } peers { permanent_uuid: "5b38e3ee13f54866a19b1be0ff824b83" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } } }
I20250904 19:58:06.296572 12991 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:06.296216 12884 ts_manager.cc:194] Registered new tserver with Master: 10f860a8c34748d09ee747a660374f2a (127.10.24.129:44773)
I20250904 19:58:06.299578 13105 heartbeater.cc:344] Connected to a master server at 127.10.24.190:35721
I20250904 19:58:06.293249 13011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f68358e98174dbe96102699d238b1fe. Latest consensus state: current_term: 1 leader_uuid: "3f68358e98174dbe96102699d238b1fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f68358e98174dbe96102699d238b1fe" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } } peers { permanent_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } } peers { permanent_uuid: "5b38e3ee13f54866a19b1be0ff824b83" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } } }
I20250904 19:58:06.300026 13105 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:06.300977 13105 heartbeater.cc:507] Master 127.10.24.190:35721 requested a full tablet report, sending...
I20250904 19:58:06.301460 13011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:06.303612 12819 ts_manager.cc:194] Registered new tserver with Master: 10f860a8c34748d09ee747a660374f2a (127.10.24.129:44773)
I20250904 19:58:06.303500 13092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3f68358e98174dbe96102699d238b1fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f68358e98174dbe96102699d238b1fe" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } } peers { permanent_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } } peers { permanent_uuid: "5b38e3ee13f54866a19b1be0ff824b83" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } } }
I20250904 19:58:06.304246 13092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:06.304893 13092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3f68358e98174dbe96102699d238b1fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f68358e98174dbe96102699d238b1fe" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } } peers { permanent_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } } peers { permanent_uuid: "5b38e3ee13f54866a19b1be0ff824b83" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } } }
I20250904 19:58:06.305281 13102 catalog_manager.cc:1349] Generated new cluster ID: 62c36856ce3d40079517efd57998e1a6
I20250904 19:58:06.305529 13092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:06.305635 13102 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 19:58:06.314966 12991 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3f68358e98174dbe96102699d238b1fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f68358e98174dbe96102699d238b1fe" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } } peers { permanent_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } } peers { permanent_uuid: "5b38e3ee13f54866a19b1be0ff824b83" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } } }
I20250904 19:58:06.315654 12991 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:06.323634 10338 webserver.cc:480] Webserver started at http://127.10.24.130:39879/ using document root <none> and password file <none>
I20250904 19:58:06.324357 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:06.324595 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:06.324903 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:06.326231 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/ts-1-root/instance:
uuid: "d14aa2e8446b45c79eff8aabee599b31"
format_stamp: "Formatted at 2025-09-04 19:58:06 on dist-test-slave-f3xj"
I20250904 19:58:06.327770 13011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3f68358e98174dbe96102699d238b1fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f68358e98174dbe96102699d238b1fe" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 35721 } } peers { permanent_uuid: "fe26647b97c74957a7d3a27c19b6fe0d" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 36539 } } peers { permanent_uuid: "5b38e3ee13f54866a19b1be0ff824b83" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 38349 } } }
I20250904 19:58:06.328444 13011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:06.338932 13102 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 19:58:06.339114 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.012s user 0.009s sys 0.004s
I20250904 19:58:06.340786 13102 catalog_manager.cc:1506] Loading token signing keys...
I20250904 19:58:06.345299 13126 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:06.346565 10338 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20250904 19:58:06.346911 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/ts-1-root
uuid: "d14aa2e8446b45c79eff8aabee599b31"
format_stamp: "Formatted at 2025-09-04 19:58:06 on dist-test-slave-f3xj"
I20250904 19:58:06.347218 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:06.369496 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:06.370800 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:06.371439 13102 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe: Generated new TSK 0
I20250904 19:58:06.374485 13102 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 19:58:06.377627 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:06.380064 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:06.380254 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:06.380533 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:06.380743 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:06.388036 12819 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53968
I20250904 19:58:06.400528 13105 heartbeater.cc:499] Master 127.10.24.190:35721 was elected leader, sending a full tablet report...
I20250904 19:58:06.424578 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.130:41545
I20250904 19:58:06.424675 13188 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.130:41545 every 8 connection(s)
I20250904 19:58:06.445423 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:06.450901 13190 heartbeater.cc:344] Connected to a master server at 127.10.24.189:36539
I20250904 19:58:06.451334 13190 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:06.452261 13190 heartbeater.cc:507] Master 127.10.24.189:36539 requested a full tablet report, sending...
W20250904 19:58:06.453734 13196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:06.454602 13189 heartbeater.cc:344] Connected to a master server at 127.10.24.188:38349
I20250904 19:58:06.455000 12884 ts_manager.cc:194] Registered new tserver with Master: d14aa2e8446b45c79eff8aabee599b31 (127.10.24.130:41545)
I20250904 19:58:06.454977 13189 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:06.456161 13189 heartbeater.cc:507] Master 127.10.24.188:38349 requested a full tablet report, sending...
W20250904 19:58:06.458745 13197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:06.459050 12950 ts_manager.cc:194] Registered new tserver with Master: d14aa2e8446b45c79eff8aabee599b31 (127.10.24.130:41545)
I20250904 19:58:06.467823 13192 heartbeater.cc:344] Connected to a master server at 127.10.24.190:35721
I20250904 19:58:06.468171 13192 heartbeater.cc:461] Registering TS with master...
W20250904 19:58:06.468387 13199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:06.469055 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:06.469023 13192 heartbeater.cc:507] Master 127.10.24.190:35721 requested a full tablet report, sending...
I20250904 19:58:06.470182 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:06.470424 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:06.470639 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015886470622 us; error 0 us; skew 500 ppm
I20250904 19:58:06.471148 12819 ts_manager.cc:194] Registered new tserver with Master: d14aa2e8446b45c79eff8aabee599b31 (127.10.24.130:41545)
I20250904 19:58:06.471205 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:06.472659 12819 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53980
I20250904 19:58:06.473706 10338 webserver.cc:480] Webserver started at http://127.10.24.131:36945/ using document root <none> and password file <none>
I20250904 19:58:06.474190 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:06.474355 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:06.474587 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:06.475772 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/ts-2-root/instance:
uuid: "9b36aa80522f4b6db08e8b332aacd3fb"
format_stamp: "Formatted at 2025-09-04 19:58:06 on dist-test-slave-f3xj"
I20250904 19:58:06.481210 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20250904 19:58:06.484243 13204 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:06.484651 13192 heartbeater.cc:499] Master 127.10.24.190:35721 was elected leader, sending a full tablet report...
I20250904 19:58:06.485020 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250904 19:58:06.485338 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/ts-2-root
uuid: "9b36aa80522f4b6db08e8b332aacd3fb"
format_stamp: "Formatted at 2025-09-04 19:58:06 on dist-test-slave-f3xj"
I20250904 19:58:06.485608 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksum.1757015869512910-10338-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:06.542976 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:06.544296 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:06.546118 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:06.549152 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:06.549407 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:06.549677 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:06.549897 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:06.603138 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.131:38275
I20250904 19:58:06.604293 13266 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.131:38275 every 8 connection(s)
I20250904 19:58:06.623163 13268 heartbeater.cc:344] Connected to a master server at 127.10.24.189:36539
I20250904 19:58:06.623690 13268 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:06.624568 13268 heartbeater.cc:507] Master 127.10.24.189:36539 requested a full tablet report, sending...
I20250904 19:58:06.627347 12883 ts_manager.cc:194] Registered new tserver with Master: 9b36aa80522f4b6db08e8b332aacd3fb (127.10.24.131:38275)
I20250904 19:58:06.629563 13267 heartbeater.cc:344] Connected to a master server at 127.10.24.188:38349
I20250904 19:58:06.630007 13267 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:06.634110 13267 heartbeater.cc:507] Master 127.10.24.188:38349 requested a full tablet report, sending...
I20250904 19:58:06.636591 12949 ts_manager.cc:194] Registered new tserver with Master: 9b36aa80522f4b6db08e8b332aacd3fb (127.10.24.131:38275)
I20250904 19:58:06.646595 13269 heartbeater.cc:344] Connected to a master server at 127.10.24.190:35721
I20250904 19:58:06.646921 13269 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:06.647615 13269 heartbeater.cc:507] Master 127.10.24.190:35721 requested a full tablet report, sending...
I20250904 19:58:06.649415 12819 ts_manager.cc:194] Registered new tserver with Master: 9b36aa80522f4b6db08e8b332aacd3fb (127.10.24.131:38275)
I20250904 19:58:06.649595 10338 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.007069477s
I20250904 19:58:06.651156 12819 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53996
I20250904 19:58:06.664572 13269 heartbeater.cc:499] Master 127.10.24.190:35721 was elected leader, sending a full tablet report...
I20250904 19:58:06.700824 12818 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53998:
name: "ksck-test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\n\000\000\000\004\001\024\000\000\000""\004\001\n\000\000\000\004\001\024\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 19:58:06.703267 12818 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table ksck-test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 19:58:06.817498 13065 tablet_service.cc:1468] Processing CreateTablet for tablet ee24c4309fab4ec2bd9912fc7725ae35 (DEFAULT_TABLE table=ksck-test-table [id=ca94c38f250549e092646bfe2fcf8eee]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:06.817972 13154 tablet_service.cc:1468] Processing CreateTablet for tablet ee24c4309fab4ec2bd9912fc7725ae35 (DEFAULT_TABLE table=ksck-test-table [id=ca94c38f250549e092646bfe2fcf8eee]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:06.818884 13065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee24c4309fab4ec2bd9912fc7725ae35. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:06.819186 13154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee24c4309fab4ec2bd9912fc7725ae35. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:06.818831 13152 tablet_service.cc:1468] Processing CreateTablet for tablet c0effd02017149eabe7b6e2d677e5ee7 (DEFAULT_TABLE table=ksck-test-table [id=ca94c38f250549e092646bfe2fcf8eee]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:06.821460 13152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0effd02017149eabe7b6e2d677e5ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:06.822535 13153 tablet_service.cc:1468] Processing CreateTablet for tablet 2f33b80099e648adafa971d07130196d (DEFAULT_TABLE table=ksck-test-table [id=ca94c38f250549e092646bfe2fcf8eee]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:06.821138 13064 tablet_service.cc:1468] Processing CreateTablet for tablet 2f33b80099e648adafa971d07130196d (DEFAULT_TABLE table=ksck-test-table [id=ca94c38f250549e092646bfe2fcf8eee]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:06.834215 13064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f33b80099e648adafa971d07130196d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:06.831983 13153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f33b80099e648adafa971d07130196d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:06.889000 13063 tablet_service.cc:1468] Processing CreateTablet for tablet c0effd02017149eabe7b6e2d677e5ee7 (DEFAULT_TABLE table=ksck-test-table [id=ca94c38f250549e092646bfe2fcf8eee]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:06.890419 13063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0effd02017149eabe7b6e2d677e5ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:06.895898 13232 tablet_service.cc:1468] Processing CreateTablet for tablet ee24c4309fab4ec2bd9912fc7725ae35 (DEFAULT_TABLE table=ksck-test-table [id=ca94c38f250549e092646bfe2fcf8eee]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:06.897284 13232 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee24c4309fab4ec2bd9912fc7725ae35. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:06.900580 13231 tablet_service.cc:1468] Processing CreateTablet for tablet 2f33b80099e648adafa971d07130196d (DEFAULT_TABLE table=ksck-test-table [id=ca94c38f250549e092646bfe2fcf8eee]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:06.901976 13231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f33b80099e648adafa971d07130196d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:06.902896 13289 tablet_bootstrap.cc:492] T ee24c4309fab4ec2bd9912fc7725ae35 P 10f860a8c34748d09ee747a660374f2a: Bootstrap starting.
I20250904 19:58:06.904387 13230 tablet_service.cc:1468] Processing CreateTablet for tablet c0effd02017149eabe7b6e2d677e5ee7 (DEFAULT_TABLE table=ksck-test-table [id=ca94c38f250549e092646bfe2fcf8eee]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:06.905321 13230 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0effd02017149eabe7b6e2d677e5ee7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:06.916014 13290 tablet_bootstrap.cc:492] T c0effd02017149eabe7b6e2d677e5ee7 P d14aa2e8446b45c79eff8aabee599b31: Bootstrap starting.
I20250904 19:58:06.921473 13290 tablet_bootstrap.cc:654] T c0effd02017149eabe7b6e2d677e5ee7 P d14aa2e8446b45c79eff8aabee599b31: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:06.932044 13289 tablet_bootstrap.cc:654] T ee24c4309fab4ec2bd9912fc7725ae35 P 10f860a8c34748d09ee747a660374f2a: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:06.938215 13289 tablet_bootstrap.cc:492] T ee24c4309fab4ec2bd9912fc7725ae35 P 10f860a8c34748d09ee747a660374f2a: No bootstrap required, opened a new log
I20250904 19:58:06.938621 13289 ts_tablet_manager.cc:1397] T ee24c4309fab4ec2bd9912fc7725ae35 P 10f860a8c34748d09ee747a660374f2a: Time spent bootstrapping tablet: real 0.036s user 0.006s sys 0.007s
I20250904 19:58:06.941258 13289 raft_consensus.cc:357] T ee24c4309fab4ec2bd9912fc7725ae35 P 10f860a8c34748d09ee747a660374f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:06.941713 13290 tablet_bootstrap.cc:492] T c0effd02017149eabe7b6e2d677e5ee7 P d14aa2e8446b45c79eff8aabee599b31: No bootstrap required, opened a new log
I20250904 19:58:06.942410 13290 ts_tablet_manager.cc:1397] T c0effd02017149eabe7b6e2d677e5ee7 P d14aa2e8446b45c79eff8aabee599b31: Time spent bootstrapping tablet: real 0.027s user 0.003s sys 0.006s
I20250904 19:58:06.941975 13289 raft_consensus.cc:383] T ee24c4309fab4ec2bd9912fc7725ae35 P 10f860a8c34748d09ee747a660374f2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:06.943207 13289 raft_consensus.cc:738] T ee24c4309fab4ec2bd9912fc7725ae35 P 10f860a8c34748d09ee747a660374f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10f860a8c34748d09ee747a660374f2a, State: Initialized, Role: FOLLOWER
I20250904 19:58:06.945546 13289 consensus_queue.cc:260] T ee24c4309fab4ec2bd9912fc7725ae35 P 10f860a8c34748d09ee747a660374f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:06.947383 13290 raft_consensus.cc:357] T c0effd02017149eabe7b6e2d677e5ee7 P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:06.953333 13290 raft_consensus.cc:383] T c0effd02017149eabe7b6e2d677e5ee7 P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:06.953678 13290 raft_consensus.cc:738] T c0effd02017149eabe7b6e2d677e5ee7 P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d14aa2e8446b45c79eff8aabee599b31, State: Initialized, Role: FOLLOWER
I20250904 19:58:06.975735 13289 ts_tablet_manager.cc:1428] T ee24c4309fab4ec2bd9912fc7725ae35 P 10f860a8c34748d09ee747a660374f2a: Time spent starting tablet: real 0.037s user 0.022s sys 0.013s
I20250904 19:58:06.968127 13290 consensus_queue.cc:260] T c0effd02017149eabe7b6e2d677e5ee7 P d14aa2e8446b45c79eff8aabee599b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:06.982574 13289 tablet_bootstrap.cc:492] T 2f33b80099e648adafa971d07130196d P 10f860a8c34748d09ee747a660374f2a: Bootstrap starting.
I20250904 19:58:06.990168 13289 tablet_bootstrap.cc:654] T 2f33b80099e648adafa971d07130196d P 10f860a8c34748d09ee747a660374f2a: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:07.009801 13294 tablet_bootstrap.cc:492] T ee24c4309fab4ec2bd9912fc7725ae35 P 9b36aa80522f4b6db08e8b332aacd3fb: Bootstrap starting.
I20250904 19:58:07.015125 13290 ts_tablet_manager.cc:1428] T c0effd02017149eabe7b6e2d677e5ee7 P d14aa2e8446b45c79eff8aabee599b31: Time spent starting tablet: real 0.072s user 0.009s sys 0.009s
I20250904 19:58:07.016137 13289 tablet_bootstrap.cc:492] T 2f33b80099e648adafa971d07130196d P 10f860a8c34748d09ee747a660374f2a: No bootstrap required, opened a new log
I20250904 19:58:07.016700 13289 ts_tablet_manager.cc:1397] T 2f33b80099e648adafa971d07130196d P 10f860a8c34748d09ee747a660374f2a: Time spent bootstrapping tablet: real 0.034s user 0.020s sys 0.010s
I20250904 19:58:07.016970 13294 tablet_bootstrap.cc:654] T ee24c4309fab4ec2bd9912fc7725ae35 P 9b36aa80522f4b6db08e8b332aacd3fb: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:07.021045 13289 raft_consensus.cc:357] T 2f33b80099e648adafa971d07130196d P 10f860a8c34748d09ee747a660374f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.021855 13289 raft_consensus.cc:383] T 2f33b80099e648adafa971d07130196d P 10f860a8c34748d09ee747a660374f2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:07.022203 13289 raft_consensus.cc:738] T 2f33b80099e648adafa971d07130196d P 10f860a8c34748d09ee747a660374f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10f860a8c34748d09ee747a660374f2a, State: Initialized, Role: FOLLOWER
I20250904 19:58:07.022917 13289 consensus_queue.cc:260] T 2f33b80099e648adafa971d07130196d P 10f860a8c34748d09ee747a660374f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.025499 13289 ts_tablet_manager.cc:1428] T 2f33b80099e648adafa971d07130196d P 10f860a8c34748d09ee747a660374f2a: Time spent starting tablet: real 0.008s user 0.001s sys 0.004s
I20250904 19:58:07.027525 13294 tablet_bootstrap.cc:492] T ee24c4309fab4ec2bd9912fc7725ae35 P 9b36aa80522f4b6db08e8b332aacd3fb: No bootstrap required, opened a new log
I20250904 19:58:07.027753 13289 tablet_bootstrap.cc:492] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a: Bootstrap starting.
I20250904 19:58:07.028137 13294 ts_tablet_manager.cc:1397] T ee24c4309fab4ec2bd9912fc7725ae35 P 9b36aa80522f4b6db08e8b332aacd3fb: Time spent bootstrapping tablet: real 0.019s user 0.011s sys 0.006s
I20250904 19:58:07.030831 13294 raft_consensus.cc:357] T ee24c4309fab4ec2bd9912fc7725ae35 P 9b36aa80522f4b6db08e8b332aacd3fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.031582 13294 raft_consensus.cc:383] T ee24c4309fab4ec2bd9912fc7725ae35 P 9b36aa80522f4b6db08e8b332aacd3fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:07.031919 13294 raft_consensus.cc:738] T ee24c4309fab4ec2bd9912fc7725ae35 P 9b36aa80522f4b6db08e8b332aacd3fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b36aa80522f4b6db08e8b332aacd3fb, State: Initialized, Role: FOLLOWER
I20250904 19:58:07.032644 13294 consensus_queue.cc:260] T ee24c4309fab4ec2bd9912fc7725ae35 P 9b36aa80522f4b6db08e8b332aacd3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.033437 13289 tablet_bootstrap.cc:654] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:07.044277 13289 tablet_bootstrap.cc:492] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a: No bootstrap required, opened a new log
I20250904 19:58:07.044740 13289 ts_tablet_manager.cc:1397] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.007s
I20250904 19:58:07.047360 13289 raft_consensus.cc:357] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.048161 13289 raft_consensus.cc:383] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:07.048446 13289 raft_consensus.cc:738] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10f860a8c34748d09ee747a660374f2a, State: Initialized, Role: FOLLOWER
I20250904 19:58:07.049278 13289 consensus_queue.cc:260] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.051971 13289 ts_tablet_manager.cc:1428] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
I20250904 19:58:07.058357 13290 tablet_bootstrap.cc:492] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31: Bootstrap starting.
I20250904 19:58:07.063890 13290 tablet_bootstrap.cc:654] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:07.064901 13294 ts_tablet_manager.cc:1428] T ee24c4309fab4ec2bd9912fc7725ae35 P 9b36aa80522f4b6db08e8b332aacd3fb: Time spent starting tablet: real 0.036s user 0.008s sys 0.013s
I20250904 19:58:07.067272 13294 tablet_bootstrap.cc:492] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb: Bootstrap starting.
I20250904 19:58:07.076308 13294 tablet_bootstrap.cc:654] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:07.107640 13290 tablet_bootstrap.cc:492] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31: No bootstrap required, opened a new log
I20250904 19:58:07.107947 13294 tablet_bootstrap.cc:492] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb: No bootstrap required, opened a new log
I20250904 19:58:07.108165 13290 ts_tablet_manager.cc:1397] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31: Time spent bootstrapping tablet: real 0.050s user 0.022s sys 0.027s
I20250904 19:58:07.108429 13294 ts_tablet_manager.cc:1397] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb: Time spent bootstrapping tablet: real 0.041s user 0.011s sys 0.017s
I20250904 19:58:07.110847 13290 raft_consensus.cc:357] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.111197 13294 raft_consensus.cc:357] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.111627 13290 raft_consensus.cc:383] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:07.111898 13294 raft_consensus.cc:383] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:07.111949 13290 raft_consensus.cc:738] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d14aa2e8446b45c79eff8aabee599b31, State: Initialized, Role: FOLLOWER
I20250904 19:58:07.112298 13294 raft_consensus.cc:738] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b36aa80522f4b6db08e8b332aacd3fb, State: Initialized, Role: FOLLOWER
I20250904 19:58:07.112761 13290 consensus_queue.cc:260] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.115113 13290 ts_tablet_manager.cc:1428] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250904 19:58:07.113161 13294 consensus_queue.cc:260] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.116096 13290 tablet_bootstrap.cc:492] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31: Bootstrap starting.
I20250904 19:58:07.117754 13294 ts_tablet_manager.cc:1428] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20250904 19:58:07.118755 13294 tablet_bootstrap.cc:492] T 2f33b80099e648adafa971d07130196d P 9b36aa80522f4b6db08e8b332aacd3fb: Bootstrap starting.
I20250904 19:58:07.121744 13290 tablet_bootstrap.cc:654] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:07.131863 13294 tablet_bootstrap.cc:654] T 2f33b80099e648adafa971d07130196d P 9b36aa80522f4b6db08e8b332aacd3fb: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:07.137015 13002 catalog_manager.cc:1261] Loaded cluster ID: 62c36856ce3d40079517efd57998e1a6
I20250904 19:58:07.137355 13002 catalog_manager.cc:1554] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d: loading cluster ID for follower catalog manager: success
I20250904 19:58:07.161048 13290 tablet_bootstrap.cc:492] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31: No bootstrap required, opened a new log
I20250904 19:58:07.161182 13294 tablet_bootstrap.cc:492] T 2f33b80099e648adafa971d07130196d P 9b36aa80522f4b6db08e8b332aacd3fb: No bootstrap required, opened a new log
I20250904 19:58:07.161628 13290 ts_tablet_manager.cc:1397] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31: Time spent bootstrapping tablet: real 0.046s user 0.021s sys 0.017s
I20250904 19:58:07.161732 13294 ts_tablet_manager.cc:1397] T 2f33b80099e648adafa971d07130196d P 9b36aa80522f4b6db08e8b332aacd3fb: Time spent bootstrapping tablet: real 0.043s user 0.021s sys 0.006s
I20250904 19:58:07.164711 13290 raft_consensus.cc:357] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.165397 13290 raft_consensus.cc:383] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:07.165671 13290 raft_consensus.cc:738] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d14aa2e8446b45c79eff8aabee599b31, State: Initialized, Role: FOLLOWER
I20250904 19:58:07.166414 13290 consensus_queue.cc:260] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.170080 13002 catalog_manager.cc:1576] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d: acquiring CA information for follower catalog manager: success
I20250904 19:58:07.171139 13290 ts_tablet_manager.cc:1428] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31: Time spent starting tablet: real 0.009s user 0.004s sys 0.002s
I20250904 19:58:07.173184 13028 catalog_manager.cc:1261] Loaded cluster ID: 62c36856ce3d40079517efd57998e1a6
I20250904 19:58:07.173463 13028 catalog_manager.cc:1554] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83: loading cluster ID for follower catalog manager: success
I20250904 19:58:07.182464 13294 raft_consensus.cc:357] T 2f33b80099e648adafa971d07130196d P 9b36aa80522f4b6db08e8b332aacd3fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.183226 13294 raft_consensus.cc:383] T 2f33b80099e648adafa971d07130196d P 9b36aa80522f4b6db08e8b332aacd3fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:07.183506 13294 raft_consensus.cc:738] T 2f33b80099e648adafa971d07130196d P 9b36aa80522f4b6db08e8b332aacd3fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b36aa80522f4b6db08e8b332aacd3fb, State: Initialized, Role: FOLLOWER
I20250904 19:58:07.184173 13028 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83: acquiring CA information for follower catalog manager: success
I20250904 19:58:07.184818 13294 consensus_queue.cc:260] T 2f33b80099e648adafa971d07130196d P 9b36aa80522f4b6db08e8b332aacd3fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.187263 13002 catalog_manager.cc:1604] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:58:07.193532 13294 ts_tablet_manager.cc:1428] T 2f33b80099e648adafa971d07130196d P 9b36aa80522f4b6db08e8b332aacd3fb: Time spent starting tablet: real 0.031s user 0.003s sys 0.003s
I20250904 19:58:07.195731 13293 raft_consensus.cc:491] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:07.196197 13293 raft_consensus.cc:513] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.198484 13293 leader_election.cc:290] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d14aa2e8446b45c79eff8aabee599b31 (127.10.24.130:41545), 9b36aa80522f4b6db08e8b332aacd3fb (127.10.24.131:38275)
I20250904 19:58:07.209538 13028 catalog_manager.cc:1604] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:58:07.221197 13295 raft_consensus.cc:491] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:07.221733 13295 raft_consensus.cc:513] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.224061 13295 leader_election.cc:290] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10f860a8c34748d09ee747a660374f2a (127.10.24.129:44773), 9b36aa80522f4b6db08e8b332aacd3fb (127.10.24.131:38275)
I20250904 19:58:07.241952 13164 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c0effd02017149eabe7b6e2d677e5ee7" candidate_uuid: "10f860a8c34748d09ee747a660374f2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14aa2e8446b45c79eff8aabee599b31" is_pre_election: true
I20250904 19:58:07.242682 13164 raft_consensus.cc:2466] T c0effd02017149eabe7b6e2d677e5ee7 P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10f860a8c34748d09ee747a660374f2a in term 0.
I20250904 19:58:07.246114 13075 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2f33b80099e648adafa971d07130196d" candidate_uuid: "d14aa2e8446b45c79eff8aabee599b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10f860a8c34748d09ee747a660374f2a" is_pre_election: true
I20250904 19:58:07.246762 13075 raft_consensus.cc:2466] T 2f33b80099e648adafa971d07130196d P 10f860a8c34748d09ee747a660374f2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d14aa2e8446b45c79eff8aabee599b31 in term 0.
I20250904 19:58:07.247994 13039 leader_election.cc:304] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10f860a8c34748d09ee747a660374f2a, d14aa2e8446b45c79eff8aabee599b31; no voters:
I20250904 19:58:07.247964 13128 leader_election.cc:304] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10f860a8c34748d09ee747a660374f2a, d14aa2e8446b45c79eff8aabee599b31; no voters:
I20250904 19:58:07.248939 13293 raft_consensus.cc:2802] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:07.249059 13295 raft_consensus.cc:2802] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:07.249349 13293 raft_consensus.cc:491] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:07.249475 13295 raft_consensus.cc:491] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:07.249851 13295 raft_consensus.cc:3058] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:07.249737 13293 raft_consensus.cc:3058] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:07.257072 13295 raft_consensus.cc:513] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.260684 13295 leader_election.cc:290] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [CANDIDATE]: Term 1 election: Requested vote from peers 10f860a8c34748d09ee747a660374f2a (127.10.24.129:44773), 9b36aa80522f4b6db08e8b332aacd3fb (127.10.24.131:38275)
I20250904 19:58:07.261895 13075 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2f33b80099e648adafa971d07130196d" candidate_uuid: "d14aa2e8446b45c79eff8aabee599b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10f860a8c34748d09ee747a660374f2a"
I20250904 19:58:07.262399 13075 raft_consensus.cc:3058] T 2f33b80099e648adafa971d07130196d P 10f860a8c34748d09ee747a660374f2a [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:07.269392 13075 raft_consensus.cc:2466] T 2f33b80099e648adafa971d07130196d P 10f860a8c34748d09ee747a660374f2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d14aa2e8446b45c79eff8aabee599b31 in term 1.
I20250904 19:58:07.270467 13128 leader_election.cc:304] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10f860a8c34748d09ee747a660374f2a, d14aa2e8446b45c79eff8aabee599b31; no voters:
I20250904 19:58:07.270933 13298 raft_consensus.cc:491] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:07.271647 13295 raft_consensus.cc:2802] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:07.271423 13298 raft_consensus.cc:513] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.272790 13242 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2f33b80099e648adafa971d07130196d" candidate_uuid: "d14aa2e8446b45c79eff8aabee599b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" is_pre_election: true
I20250904 19:58:07.273439 13242 raft_consensus.cc:2466] T 2f33b80099e648adafa971d07130196d P 9b36aa80522f4b6db08e8b332aacd3fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d14aa2e8446b45c79eff8aabee599b31 in term 0.
I20250904 19:58:07.274019 13298 leader_election.cc:290] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10f860a8c34748d09ee747a660374f2a (127.10.24.129:44773), d14aa2e8446b45c79eff8aabee599b31 (127.10.24.130:41545)
I20250904 19:58:07.274441 13241 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c0effd02017149eabe7b6e2d677e5ee7" candidate_uuid: "10f860a8c34748d09ee747a660374f2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" is_pre_election: true
I20250904 19:58:07.275044 13241 raft_consensus.cc:2466] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 10f860a8c34748d09ee747a660374f2a in term 0.
I20250904 19:58:07.275882 13293 raft_consensus.cc:513] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.276538 13241 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2f33b80099e648adafa971d07130196d" candidate_uuid: "d14aa2e8446b45c79eff8aabee599b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b36aa80522f4b6db08e8b332aacd3fb"
I20250904 19:58:07.277102 13241 raft_consensus.cc:3058] T 2f33b80099e648adafa971d07130196d P 9b36aa80522f4b6db08e8b332aacd3fb [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:07.282342 13293 leader_election.cc:290] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [CANDIDATE]: Term 1 election: Requested vote from peers d14aa2e8446b45c79eff8aabee599b31 (127.10.24.130:41545), 9b36aa80522f4b6db08e8b332aacd3fb (127.10.24.131:38275)
I20250904 19:58:07.287163 13241 raft_consensus.cc:2466] T 2f33b80099e648adafa971d07130196d P 9b36aa80522f4b6db08e8b332aacd3fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d14aa2e8446b45c79eff8aabee599b31 in term 1.
I20250904 19:58:07.291198 13164 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c0effd02017149eabe7b6e2d677e5ee7" candidate_uuid: "10f860a8c34748d09ee747a660374f2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14aa2e8446b45c79eff8aabee599b31"
I20250904 19:58:07.291858 13164 raft_consensus.cc:3058] T c0effd02017149eabe7b6e2d677e5ee7 P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:07.294050 13241 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c0effd02017149eabe7b6e2d677e5ee7" candidate_uuid: "10f860a8c34748d09ee747a660374f2a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b36aa80522f4b6db08e8b332aacd3fb"
I20250904 19:58:07.294653 13241 raft_consensus.cc:3058] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:07.303061 13241 raft_consensus.cc:2466] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10f860a8c34748d09ee747a660374f2a in term 1.
I20250904 19:58:07.304041 13039 leader_election.cc:304] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10f860a8c34748d09ee747a660374f2a, 9b36aa80522f4b6db08e8b332aacd3fb; no voters:
I20250904 19:58:07.304994 13293 raft_consensus.cc:2802] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:07.307989 13164 raft_consensus.cc:2466] T c0effd02017149eabe7b6e2d677e5ee7 P d14aa2e8446b45c79eff8aabee599b31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 10f860a8c34748d09ee747a660374f2a in term 1.
I20250904 19:58:07.315199 13312 raft_consensus.cc:491] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:07.314595 13295 raft_consensus.cc:695] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [term 1 LEADER]: Becoming Leader. State: Replica: d14aa2e8446b45c79eff8aabee599b31, State: Running, Role: LEADER
I20250904 19:58:07.316890 13295 consensus_queue.cc:237] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.315711 13312 raft_consensus.cc:513] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.322901 13074 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "ee24c4309fab4ec2bd9912fc7725ae35" candidate_uuid: "d14aa2e8446b45c79eff8aabee599b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10f860a8c34748d09ee747a660374f2a" is_pre_election: true
I20250904 19:58:07.323554 13074 raft_consensus.cc:2466] T ee24c4309fab4ec2bd9912fc7725ae35 P 10f860a8c34748d09ee747a660374f2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d14aa2e8446b45c79eff8aabee599b31 in term 0.
I20250904 19:58:07.324505 13128 leader_election.cc:304] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10f860a8c34748d09ee747a660374f2a, d14aa2e8446b45c79eff8aabee599b31; no voters:
I20250904 19:58:07.324738 13075 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c0effd02017149eabe7b6e2d677e5ee7" candidate_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10f860a8c34748d09ee747a660374f2a" is_pre_election: true
I20250904 19:58:07.326112 13241 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "ee24c4309fab4ec2bd9912fc7725ae35" candidate_uuid: "d14aa2e8446b45c79eff8aabee599b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" is_pre_election: true
I20250904 19:58:07.326260 13309 raft_consensus.cc:2802] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:07.326695 13309 raft_consensus.cc:491] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:07.327414 13309 raft_consensus.cc:3058] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:07.326745 13241 raft_consensus.cc:2466] T ee24c4309fab4ec2bd9912fc7725ae35 P 9b36aa80522f4b6db08e8b332aacd3fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d14aa2e8446b45c79eff8aabee599b31 in term 0.
I20250904 19:58:07.330881 13293 raft_consensus.cc:695] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [term 1 LEADER]: Becoming Leader. State: Replica: 10f860a8c34748d09ee747a660374f2a, State: Running, Role: LEADER
I20250904 19:58:07.332273 13293 consensus_queue.cc:237] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.334786 13309 raft_consensus.cc:513] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.338006 13164 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c0effd02017149eabe7b6e2d677e5ee7" candidate_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14aa2e8446b45c79eff8aabee599b31" is_pre_election: true
I20250904 19:58:07.338737 13164 raft_consensus.cc:2391] T c0effd02017149eabe7b6e2d677e5ee7 P d14aa2e8446b45c79eff8aabee599b31 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9b36aa80522f4b6db08e8b332aacd3fb in current term 1: Already voted for candidate 10f860a8c34748d09ee747a660374f2a in this term.
I20250904 19:58:07.337956 13312 leader_election.cc:290] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10f860a8c34748d09ee747a660374f2a (127.10.24.129:44773), 9b36aa80522f4b6db08e8b332aacd3fb (127.10.24.131:38275)
I20250904 19:58:07.342227 13074 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "ee24c4309fab4ec2bd9912fc7725ae35" candidate_uuid: "d14aa2e8446b45c79eff8aabee599b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10f860a8c34748d09ee747a660374f2a"
I20250904 19:58:07.342803 13074 raft_consensus.cc:3058] T ee24c4309fab4ec2bd9912fc7725ae35 P 10f860a8c34748d09ee747a660374f2a [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:07.344318 13206 leader_election.cc:304] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b36aa80522f4b6db08e8b332aacd3fb; no voters: 10f860a8c34748d09ee747a660374f2a, d14aa2e8446b45c79eff8aabee599b31
I20250904 19:58:07.345168 13298 raft_consensus.cc:2747] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250904 19:58:07.348222 12819 catalog_manager.cc:5582] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a reported cstate change: term changed from 0 to 1, leader changed from <none> to 10f860a8c34748d09ee747a660374f2a (127.10.24.129). New cstate: current_term: 1 leader_uuid: "10f860a8c34748d09ee747a660374f2a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } health_report { overall_health: UNKNOWN } } }
I20250904 19:58:07.349305 13241 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "ee24c4309fab4ec2bd9912fc7725ae35" candidate_uuid: "d14aa2e8446b45c79eff8aabee599b31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b36aa80522f4b6db08e8b332aacd3fb"
I20250904 19:58:07.349862 13241 raft_consensus.cc:3058] T ee24c4309fab4ec2bd9912fc7725ae35 P 9b36aa80522f4b6db08e8b332aacd3fb [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:07.353300 12817 catalog_manager.cc:5582] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 reported cstate change: term changed from 0 to 1, leader changed from <none> to d14aa2e8446b45c79eff8aabee599b31 (127.10.24.130). New cstate: current_term: 1 leader_uuid: "d14aa2e8446b45c79eff8aabee599b31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } health_report { overall_health: UNKNOWN } } }
I20250904 19:58:07.357231 13074 raft_consensus.cc:2466] T ee24c4309fab4ec2bd9912fc7725ae35 P 10f860a8c34748d09ee747a660374f2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d14aa2e8446b45c79eff8aabee599b31 in term 1.
I20250904 19:58:07.358434 13128 leader_election.cc:304] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10f860a8c34748d09ee747a660374f2a, d14aa2e8446b45c79eff8aabee599b31; no voters:
I20250904 19:58:07.359083 13241 raft_consensus.cc:2466] T ee24c4309fab4ec2bd9912fc7725ae35 P 9b36aa80522f4b6db08e8b332aacd3fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d14aa2e8446b45c79eff8aabee599b31 in term 1.
I20250904 19:58:07.360138 13309 leader_election.cc:290] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [CANDIDATE]: Term 1 election: Requested vote from peers 10f860a8c34748d09ee747a660374f2a (127.10.24.129:44773), 9b36aa80522f4b6db08e8b332aacd3fb (127.10.24.131:38275)
I20250904 19:58:07.361075 13295 raft_consensus.cc:2802] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:07.361567 13295 raft_consensus.cc:695] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [term 1 LEADER]: Becoming Leader. State: Replica: d14aa2e8446b45c79eff8aabee599b31, State: Running, Role: LEADER
I20250904 19:58:07.362383 13295 consensus_queue.cc:237] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } }
I20250904 19:58:07.418332 12817 catalog_manager.cc:5582] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 reported cstate change: term changed from 0 to 1, leader changed from <none> to d14aa2e8446b45c79eff8aabee599b31 (127.10.24.130). New cstate: current_term: 1 leader_uuid: "d14aa2e8446b45c79eff8aabee599b31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 } health_report { overall_health: UNKNOWN } } }
I20250904 19:58:07.590318 10338 ksck_remote-test.cc:376] Generating row writes...
I20250904 19:58:07.692241 13241 raft_consensus.cc:1273] T 2f33b80099e648adafa971d07130196d P 9b36aa80522f4b6db08e8b332aacd3fb [term 1 FOLLOWER]: Refusing update from remote peer d14aa2e8446b45c79eff8aabee599b31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:07.692273 13242 raft_consensus.cc:1273] T ee24c4309fab4ec2bd9912fc7725ae35 P 9b36aa80522f4b6db08e8b332aacd3fb [term 1 FOLLOWER]: Refusing update from remote peer d14aa2e8446b45c79eff8aabee599b31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:07.694279 13074 raft_consensus.cc:1273] T 2f33b80099e648adafa971d07130196d P 10f860a8c34748d09ee747a660374f2a [term 1 FOLLOWER]: Refusing update from remote peer d14aa2e8446b45c79eff8aabee599b31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:07.694593 13075 raft_consensus.cc:1273] T ee24c4309fab4ec2bd9912fc7725ae35 P 10f860a8c34748d09ee747a660374f2a [term 1 FOLLOWER]: Refusing update from remote peer d14aa2e8446b45c79eff8aabee599b31: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:07.696538 13312 consensus_queue.cc:1035] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:07.697917 13312 consensus_queue.cc:1035] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:07.699030 13312 consensus_queue.cc:1035] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:07.700639 13164 raft_consensus.cc:1273] T c0effd02017149eabe7b6e2d677e5ee7 P d14aa2e8446b45c79eff8aabee599b31 [term 1 FOLLOWER]: Refusing update from remote peer 10f860a8c34748d09ee747a660374f2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:07.704756 13293 consensus_queue.cc:1035] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d14aa2e8446b45c79eff8aabee599b31" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:07.704985 13242 raft_consensus.cc:1273] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb [term 1 FOLLOWER]: Refusing update from remote peer 10f860a8c34748d09ee747a660374f2a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:07.707544 13311 consensus_queue.cc:1035] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b36aa80522f4b6db08e8b332aacd3fb" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 38275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:07.730300 13312 consensus_queue.cc:1035] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10f860a8c34748d09ee747a660374f2a" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:08.499457 13063 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250904 19:58:08.538125 13153 tablet_service.cc:1430] Tablet server has 2 leaders and 0 scanners
I20250904 19:58:08.548913 13231 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250904 19:58:08.948695 10338 tablet_server.cc:178] TabletServer@127.10.24.129:0 shutting down...
I20250904 19:58:09.005268 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:09.006625 10338 tablet_replica.cc:331] T 2f33b80099e648adafa971d07130196d P 10f860a8c34748d09ee747a660374f2a: stopping tablet replica
I20250904 19:58:09.007450 10338 raft_consensus.cc:2241] T 2f33b80099e648adafa971d07130196d P 10f860a8c34748d09ee747a660374f2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:09.008163 10338 raft_consensus.cc:2270] T 2f33b80099e648adafa971d07130196d P 10f860a8c34748d09ee747a660374f2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:09.010138 10338 tablet_replica.cc:331] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a: stopping tablet replica
I20250904 19:58:09.010682 10338 raft_consensus.cc:2241] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:09.011595 10338 raft_consensus.cc:2270] T c0effd02017149eabe7b6e2d677e5ee7 P 10f860a8c34748d09ee747a660374f2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:09.013770 10338 tablet_replica.cc:331] T ee24c4309fab4ec2bd9912fc7725ae35 P 10f860a8c34748d09ee747a660374f2a: stopping tablet replica
I20250904 19:58:09.014425 10338 raft_consensus.cc:2241] T ee24c4309fab4ec2bd9912fc7725ae35 P 10f860a8c34748d09ee747a660374f2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:09.015034 10338 raft_consensus.cc:2270] T ee24c4309fab4ec2bd9912fc7725ae35 P 10f860a8c34748d09ee747a660374f2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:09.031889 10338 tablet_server.cc:195] TabletServer@127.10.24.129:0 shutdown complete.
I20250904 19:58:09.050999 10338 tablet_server.cc:178] TabletServer@127.10.24.130:0 shutting down...
I20250904 19:58:09.089882 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:09.090466 10338 tablet_replica.cc:331] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31: stopping tablet replica
I20250904 19:58:09.091128 10338 raft_consensus.cc:2241] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:09.092145 10338 raft_consensus.cc:2270] T 2f33b80099e648adafa971d07130196d P d14aa2e8446b45c79eff8aabee599b31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:09.094653 10338 tablet_replica.cc:331] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31: stopping tablet replica
I20250904 19:58:09.095127 10338 raft_consensus.cc:2241] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:09.095841 10338 raft_consensus.cc:2270] T ee24c4309fab4ec2bd9912fc7725ae35 P d14aa2e8446b45c79eff8aabee599b31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:09.097477 10338 tablet_replica.cc:331] T c0effd02017149eabe7b6e2d677e5ee7 P d14aa2e8446b45c79eff8aabee599b31: stopping tablet replica
I20250904 19:58:09.097918 10338 raft_consensus.cc:2241] T c0effd02017149eabe7b6e2d677e5ee7 P d14aa2e8446b45c79eff8aabee599b31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:09.098325 10338 raft_consensus.cc:2270] T c0effd02017149eabe7b6e2d677e5ee7 P d14aa2e8446b45c79eff8aabee599b31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:09.110189 10338 tablet_server.cc:195] TabletServer@127.10.24.130:0 shutdown complete.
I20250904 19:58:09.122180 10338 tablet_server.cc:178] TabletServer@127.10.24.131:0 shutting down...
I20250904 19:58:09.153324 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:09.153973 10338 tablet_replica.cc:331] T 2f33b80099e648adafa971d07130196d P 9b36aa80522f4b6db08e8b332aacd3fb: stopping tablet replica
I20250904 19:58:09.154533 10338 raft_consensus.cc:2241] T 2f33b80099e648adafa971d07130196d P 9b36aa80522f4b6db08e8b332aacd3fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:09.155004 10338 raft_consensus.cc:2270] T 2f33b80099e648adafa971d07130196d P 9b36aa80522f4b6db08e8b332aacd3fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:09.156729 10338 tablet_replica.cc:331] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb: stopping tablet replica
I20250904 19:58:09.157218 10338 raft_consensus.cc:2241] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:09.157691 10338 raft_consensus.cc:2270] T c0effd02017149eabe7b6e2d677e5ee7 P 9b36aa80522f4b6db08e8b332aacd3fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:09.159386 10338 tablet_replica.cc:331] T ee24c4309fab4ec2bd9912fc7725ae35 P 9b36aa80522f4b6db08e8b332aacd3fb: stopping tablet replica
I20250904 19:58:09.159857 10338 raft_consensus.cc:2241] T ee24c4309fab4ec2bd9912fc7725ae35 P 9b36aa80522f4b6db08e8b332aacd3fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:09.160265 10338 raft_consensus.cc:2270] T ee24c4309fab4ec2bd9912fc7725ae35 P 9b36aa80522f4b6db08e8b332aacd3fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:09.180058 10338 tablet_server.cc:195] TabletServer@127.10.24.131:0 shutdown complete.
I20250904 19:58:09.191047 10338 master.cc:561] Master@127.10.24.190:35721 shutting down...
I20250904 19:58:09.209924 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:09.210781 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:09.211222 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 3f68358e98174dbe96102699d238b1fe: stopping tablet replica
I20250904 19:58:09.232681 10338 master.cc:583] Master@127.10.24.190:35721 shutdown complete.
I20250904 19:58:09.246551 10338 master.cc:561] Master@127.10.24.189:36539 shutting down...
I20250904 19:58:09.264863 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:09.265527 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:09.265911 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P fe26647b97c74957a7d3a27c19b6fe0d: stopping tablet replica
I20250904 19:58:09.286221 10338 master.cc:583] Master@127.10.24.189:36539 shutdown complete.
I20250904 19:58:09.299835 10338 master.cc:561] Master@127.10.24.188:38349 shutting down...
I20250904 19:58:09.318269 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:09.318951 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:09.319317 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 5b38e3ee13f54866a19b1be0ff824b83: stopping tablet replica
I20250904 19:58:09.338444 10338 master.cc:583] Master@127.10.24.188:38349 shutdown complete.
[ OK ] RemoteKsckTest.TestChecksum (3559 ms)
[ RUN ] RemoteKsckTest.TestChecksumTimeout
I20250904 19:58:09.368495 10338 test_util.cc:276] Using random seed: 603149720
I20250904 19:58:09.369410 10338 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.24.190:40239,127.10.24.189:46123,127.10.24.188:44757
I20250904 19:58:09.370615 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:09.375517 13386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:09.376859 13387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:09.379072 10338 server_base.cc:1047] running on GCE node
W20250904 19:58:09.379786 13389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:09.380558 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:09.380779 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:09.380944 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015889380925 us; error 0 us; skew 500 ppm
I20250904 19:58:09.381464 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:09.383970 10338 webserver.cc:480] Webserver started at http://127.10.24.190:39561/ using document root <none> and password file <none>
I20250904 19:58:09.384402 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:09.384569 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:09.384799 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:09.385793 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/master-0-root/instance:
uuid: "f6df0c299625444596e7565436e9c042"
format_stamp: "Formatted at 2025-09-04 19:58:09 on dist-test-slave-f3xj"
I20250904 19:58:09.390060 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20250904 19:58:09.393270 13394 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:09.394052 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20250904 19:58:09.394309 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/master-0-root
uuid: "f6df0c299625444596e7565436e9c042"
format_stamp: "Formatted at 2025-09-04 19:58:09 on dist-test-slave-f3xj"
I20250904 19:58:09.394557 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:09.410002 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:09.410931 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:09.456173 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.190:40239
I20250904 19:58:09.456271 13445 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.190:40239 every 8 connection(s)
I20250904 19:58:09.460237 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:09.460510 13446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20250904 19:58:09.468654 13448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:09.470552 13446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:58:09.474987 13451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:09.476081 13449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:09.479423 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:09.481223 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:09.481565 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:09.481817 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015889481798 us; error 0 us; skew 500 ppm
I20250904 19:58:09.482583 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:09.487457 10338 webserver.cc:480] Webserver started at http://127.10.24.189:39627/ using document root <none> and password file <none>
I20250904 19:58:09.488122 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:09.488354 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:09.488646 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:09.489941 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/master-1-root/instance:
uuid: "e125eccd412340bfb2e1e817d48ebda5"
format_stamp: "Formatted at 2025-09-04 19:58:09 on dist-test-slave-f3xj"
I20250904 19:58:09.495707 13446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:09.495985 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
W20250904 19:58:09.500700 13446 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.189:46123: Network error: Client connection negotiation failed: client connection to 127.10.24.189:46123: connect: Connection refused (error 111)
I20250904 19:58:09.501067 13459 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:09.501907 10338 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20250904 19:58:09.502303 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/master-1-root
uuid: "e125eccd412340bfb2e1e817d48ebda5"
format_stamp: "Formatted at 2025-09-04 19:58:09 on dist-test-slave-f3xj"
I20250904 19:58:09.502818 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:09.530613 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:09.531963 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:09.555145 13446 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } attempt: 1
W20250904 19:58:09.560204 13446 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.189:46123: Network error: Client connection negotiation failed: client connection to 127.10.24.189:46123: connect: Connection refused (error 111)
I20250904 19:58:09.575582 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.189:46123
I20250904 19:58:09.575685 13510 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.189:46123 every 8 connection(s)
I20250904 19:58:09.580358 13511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:09.581202 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:09.586985 13513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:09.590229 13511 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:58:09.591535 13514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:09.602689 10338 server_base.cc:1047] running on GCE node
W20250904 19:58:09.604526 13517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:09.605533 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:09.605808 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:09.606007 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015889605991 us; error 0 us; skew 500 ppm
I20250904 19:58:09.606681 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:09.610324 10338 webserver.cc:480] Webserver started at http://127.10.24.188:37223/ using document root <none> and password file <none>
I20250904 19:58:09.610644 13511 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:09.610958 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:09.611259 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:09.611598 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:09.612674 13446 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } attempt: 2
I20250904 19:58:09.612972 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/master-2-root/instance:
uuid: "8c543ae5e9af4e52ab824d52f0097388"
format_stamp: "Formatted at 2025-09-04 19:58:09 on dist-test-slave-f3xj"
I20250904 19:58:09.625231 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.012s user 0.007s sys 0.004s
I20250904 19:58:09.636296 13524 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:09.637205 10338 fs_manager.cc:730] Time spent opening block manager: real 0.009s user 0.007s sys 0.001s
I20250904 19:58:09.637549 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/master-2-root
uuid: "8c543ae5e9af4e52ab824d52f0097388"
format_stamp: "Formatted at 2025-09-04 19:58:09 on dist-test-slave-f3xj"
I20250904 19:58:09.637898 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:09.643932 13511 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:09.645051 13446 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:58:09.650961 13446 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:44757: Network error: Client connection negotiation failed: client connection to 127.10.24.188:44757: connect: Connection refused (error 111)
W20250904 19:58:09.661933 13511 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:44757: Network error: Client connection negotiation failed: client connection to 127.10.24.188:44757: connect: Connection refused (error 111)
I20250904 19:58:09.672457 13446 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } attempt: 1
W20250904 19:58:09.676357 13446 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:44757: Network error: Client connection negotiation failed: client connection to 127.10.24.188:44757: connect: Connection refused (error 111)
I20250904 19:58:09.680428 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:09.681768 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:09.712888 13446 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } attempt: 2
W20250904 19:58:09.718216 13446 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:44757: Network error: Client connection negotiation failed: client connection to 127.10.24.188:44757: connect: Connection refused (error 111)
I20250904 19:58:09.746933 13511 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } attempt: 1
W20250904 19:58:09.757591 13511 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:44757: Network error: Client connection negotiation failed: client connection to 127.10.24.188:44757: connect: Connection refused (error 111)
I20250904 19:58:09.765424 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.188:44757
I20250904 19:58:09.765691 13577 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.188:44757 every 8 connection(s)
I20250904 19:58:09.772022 13578 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:09.773000 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 19:58:09.785012 13578 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:09.799347 13578 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:09.811741 13578 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:09.812260 13511 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } attempt: 2
I20250904 19:58:09.831866 13446 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } attempt: 3
I20250904 19:58:09.833195 13578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388: Bootstrap starting.
I20250904 19:58:09.837889 13511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5: Bootstrap starting.
I20250904 19:58:09.841130 13578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:09.845471 13511 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:09.848655 13578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388: No bootstrap required, opened a new log
I20250904 19:58:09.852221 13578 raft_consensus.cc:357] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6df0c299625444596e7565436e9c042" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } } peers { permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } } peers { permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } }
I20250904 19:58:09.853107 13578 raft_consensus.cc:383] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:09.853413 13578 raft_consensus.cc:738] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c543ae5e9af4e52ab824d52f0097388, State: Initialized, Role: FOLLOWER
I20250904 19:58:09.854275 13578 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6df0c299625444596e7565436e9c042" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } } peers { permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } } peers { permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } }
I20250904 19:58:09.856120 13446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042: Bootstrap starting.
I20250904 19:58:09.857829 13511 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5: No bootstrap required, opened a new log
I20250904 19:58:09.860780 13511 raft_consensus.cc:357] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6df0c299625444596e7565436e9c042" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } } peers { permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } } peers { permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } }
I20250904 19:58:09.861639 13511 raft_consensus.cc:383] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:09.862040 13511 raft_consensus.cc:738] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e125eccd412340bfb2e1e817d48ebda5, State: Initialized, Role: FOLLOWER
I20250904 19:58:09.862941 13511 consensus_queue.cc:260] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6df0c299625444596e7565436e9c042" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } } peers { permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } } peers { permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } }
I20250904 19:58:09.863873 13446 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:09.865339 13585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6df0c299625444596e7565436e9c042" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } } peers { permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } } peers { permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } } }
I20250904 19:58:09.866225 13585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:09.867223 13578 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:09.870543 13586 sys_catalog.cc:455] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6df0c299625444596e7565436e9c042" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } } peers { permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } } peers { permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } } }
I20250904 19:58:09.873971 13586 sys_catalog.cc:458] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:09.873628 13511 sys_catalog.cc:564] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:09.880299 13446 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042: No bootstrap required, opened a new log
I20250904 19:58:09.883586 13446 raft_consensus.cc:357] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6df0c299625444596e7565436e9c042" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } } peers { permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } } peers { permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } }
I20250904 19:58:09.884323 13446 raft_consensus.cc:383] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:09.884626 13446 raft_consensus.cc:738] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6df0c299625444596e7565436e9c042, State: Initialized, Role: FOLLOWER
I20250904 19:58:09.885532 13446 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6df0c299625444596e7565436e9c042" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } } peers { permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } } peers { permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } }
I20250904 19:58:09.900985 13592 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6df0c299625444596e7565436e9c042" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } } peers { permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } } peers { permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } } }
I20250904 19:58:09.901816 13592 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:09.902904 13446 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:09.937832 13592 raft_consensus.cc:491] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:09.938467 13592 raft_consensus.cc:513] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6df0c299625444596e7565436e9c042" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } } peers { permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } } peers { permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } }
I20250904 19:58:09.938990 13610 catalog_manager.cc:797] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20250904 19:58:09.942184 13551 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f6df0c299625444596e7565436e9c042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c543ae5e9af4e52ab824d52f0097388" is_pre_election: true
I20250904 19:58:09.943020 13551 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6df0c299625444596e7565436e9c042 in term 0.
I20250904 19:58:09.943987 13592 leader_election.cc:290] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e125eccd412340bfb2e1e817d48ebda5 (127.10.24.189:46123), 8c543ae5e9af4e52ab824d52f0097388 (127.10.24.188:44757)
I20250904 19:58:09.946045 13486 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f6df0c299625444596e7565436e9c042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e125eccd412340bfb2e1e817d48ebda5" is_pre_election: true
I20250904 19:58:09.946792 13486 raft_consensus.cc:2466] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6df0c299625444596e7565436e9c042 in term 0.
I20250904 19:58:09.948400 13398 leader_election.cc:304] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c543ae5e9af4e52ab824d52f0097388, f6df0c299625444596e7565436e9c042; no voters:
I20250904 19:58:09.950372 13592 raft_consensus.cc:2802] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:09.950758 13592 raft_consensus.cc:491] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:09.951085 13592 raft_consensus.cc:3058] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:09.960197 13592 raft_consensus.cc:513] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6df0c299625444596e7565436e9c042" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } } peers { permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } } peers { permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } }
I20250904 19:58:09.963326 13486 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f6df0c299625444596e7565436e9c042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e125eccd412340bfb2e1e817d48ebda5"
I20250904 19:58:09.963963 13486 raft_consensus.cc:3058] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:09.972074 13486 raft_consensus.cc:2466] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6df0c299625444596e7565436e9c042 in term 1.
I20250904 19:58:09.974848 13551 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f6df0c299625444596e7565436e9c042" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c543ae5e9af4e52ab824d52f0097388"
I20250904 19:58:09.975531 13551 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:09.983467 13551 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6df0c299625444596e7565436e9c042 in term 1.
I20250904 19:58:09.984879 13398 leader_election.cc:304] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c543ae5e9af4e52ab824d52f0097388, f6df0c299625444596e7565436e9c042; no voters:
I20250904 19:58:09.991571 13592 leader_election.cc:290] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [CANDIDATE]: Term 1 election: Requested vote from peers e125eccd412340bfb2e1e817d48ebda5 (127.10.24.189:46123), 8c543ae5e9af4e52ab824d52f0097388 (127.10.24.188:44757)
I20250904 19:58:09.992868 13592 raft_consensus.cc:2802] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:09.994035 13592 raft_consensus.cc:695] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [term 1 LEADER]: Becoming Leader. State: Replica: f6df0c299625444596e7565436e9c042, State: Running, Role: LEADER
I20250904 19:58:09.995548 13592 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6df0c299625444596e7565436e9c042" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } } peers { permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } } peers { permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } }
I20250904 19:58:10.003458 13616 catalog_manager.cc:797] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20250904 19:58:10.008829 13614 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6df0c299625444596e7565436e9c042. Latest consensus state: current_term: 1 leader_uuid: "f6df0c299625444596e7565436e9c042" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6df0c299625444596e7565436e9c042" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } } peers { permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } } peers { permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } } }
I20250904 19:58:10.009714 13614 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:10.012524 13623 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 19:58:10.016255 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20250904 19:58:10.016559 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20250904 19:58:10.017576 13623 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 19:58:10.018745 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:10.026319 13551 raft_consensus.cc:1273] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [term 1 FOLLOWER]: Refusing update from remote peer f6df0c299625444596e7565436e9c042: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:10.026969 13486 raft_consensus.cc:1273] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5 [term 1 FOLLOWER]: Refusing update from remote peer f6df0c299625444596e7565436e9c042: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:10.028131 13614 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:10.028873 13592 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250904 19:58:10.042071 13626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:10.046808 13633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:10.048133 10338 server_base.cc:1047] running on GCE node
W20250904 19:58:10.050828 13627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:10.051585 13585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6df0c299625444596e7565436e9c042. Latest consensus state: current_term: 1 leader_uuid: "f6df0c299625444596e7565436e9c042" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6df0c299625444596e7565436e9c042" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } } peers { permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } } peers { permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } } }
I20250904 19:58:10.052351 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20250904 19:58:10.052366 13585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [sys.catalog]: This master's current role is: FOLLOWER
W20250904 19:58:10.052696 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:10.053488 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015890053467 us; error 0 us; skew 500 ppm
I20250904 19:58:10.054343 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:10.060153 13592 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f6df0c299625444596e7565436e9c042" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6df0c299625444596e7565436e9c042" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } } peers { permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } } peers { permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } } }
I20250904 19:58:10.060922 13592 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:10.064615 13585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f6df0c299625444596e7565436e9c042" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6df0c299625444596e7565436e9c042" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } } peers { permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } } peers { permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } } }
I20250904 19:58:10.065399 13585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:10.067395 13586 sys_catalog.cc:455] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6df0c299625444596e7565436e9c042. Latest consensus state: current_term: 1 leader_uuid: "f6df0c299625444596e7565436e9c042" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6df0c299625444596e7565436e9c042" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } } peers { permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } } peers { permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } } }
I20250904 19:58:10.068133 13586 sys_catalog.cc:458] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:10.069136 13623 catalog_manager.cc:1349] Generated new cluster ID: 43a2c421c08b4d99b4d9382282ab8793
I20250904 19:58:10.069517 13623 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 19:58:10.072782 13614 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f6df0c299625444596e7565436e9c042" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6df0c299625444596e7565436e9c042" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } } peers { permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } } peers { permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } } }
I20250904 19:58:10.073462 13614 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:10.079854 10338 webserver.cc:480] Webserver started at http://127.10.24.129:39793/ using document root <none> and password file <none>
I20250904 19:58:10.080602 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:10.080937 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:10.081399 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:10.083173 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/ts-0-root/instance:
uuid: "566e42931fac46a68901dc3725fe14d7"
format_stamp: "Formatted at 2025-09-04 19:58:10 on dist-test-slave-f3xj"
I20250904 19:58:10.085209 13586 sys_catalog.cc:455] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f6df0c299625444596e7565436e9c042" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6df0c299625444596e7565436e9c042" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } } peers { permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } } peers { permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } } }
I20250904 19:58:10.085927 13586 sys_catalog.cc:458] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:10.092164 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20250904 19:58:10.110602 13623 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 19:58:10.112550 13623 catalog_manager.cc:1506] Loading token signing keys...
I20250904 19:58:10.118829 13641 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:10.119935 10338 fs_manager.cc:730] Time spent opening block manager: real 0.025s user 0.013s sys 0.012s
I20250904 19:58:10.120363 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/ts-0-root
uuid: "566e42931fac46a68901dc3725fe14d7"
format_stamp: "Formatted at 2025-09-04 19:58:10 on dist-test-slave-f3xj"
I20250904 19:58:10.120754 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:10.137566 13623 catalog_manager.cc:5955] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042: Generated new TSK 0
I20250904 19:58:10.138360 13623 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 19:58:10.159888 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:10.161301 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:10.162817 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:10.165695 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:10.165915 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:10.166204 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:10.166381 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:10.221820 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.129:44369
I20250904 19:58:10.221915 13704 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.129:44369 every 8 connection(s)
I20250904 19:58:10.252501 13706 heartbeater.cc:344] Connected to a master server at 127.10.24.189:46123
I20250904 19:58:10.252939 13706 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:10.253824 13706 heartbeater.cc:507] Master 127.10.24.189:46123 requested a full tablet report, sending...
I20250904 19:58:10.254534 13705 heartbeater.cc:344] Connected to a master server at 127.10.24.188:44757
I20250904 19:58:10.254901 13705 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:10.255837 13705 heartbeater.cc:507] Master 127.10.24.188:44757 requested a full tablet report, sending...
I20250904 19:58:10.256947 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:10.257014 13476 ts_manager.cc:194] Registered new tserver with Master: 566e42931fac46a68901dc3725fe14d7 (127.10.24.129:44369)
I20250904 19:58:10.258729 13542 ts_manager.cc:194] Registered new tserver with Master: 566e42931fac46a68901dc3725fe14d7 (127.10.24.129:44369)
W20250904 19:58:10.264477 13712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:10.268334 13713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:10.273515 13707 heartbeater.cc:344] Connected to a master server at 127.10.24.190:40239
I20250904 19:58:10.273962 13707 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:10.274792 13707 heartbeater.cc:507] Master 127.10.24.190:40239 requested a full tablet report, sending...
I20250904 19:58:10.277094 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:10.277767 13411 ts_manager.cc:194] Registered new tserver with Master: 566e42931fac46a68901dc3725fe14d7 (127.10.24.129:44369)
W20250904 19:58:10.278910 13715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:10.280015 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20250904 19:58:10.280066 13411 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52392
W20250904 19:58:10.280371 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:10.280619 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015890280601 us; error 0 us; skew 500 ppm
I20250904 19:58:10.281365 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:10.284266 10338 webserver.cc:480] Webserver started at http://127.10.24.130:34739/ using document root <none> and password file <none>
I20250904 19:58:10.284873 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:10.285094 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:10.285398 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:10.286798 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/ts-1-root/instance:
uuid: "ab6ec379ac4a4790b8636bc0ab141e06"
format_stamp: "Formatted at 2025-09-04 19:58:10 on dist-test-slave-f3xj"
I20250904 19:58:10.292284 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20250904 19:58:10.292644 13707 heartbeater.cc:499] Master 127.10.24.190:40239 was elected leader, sending a full tablet report...
I20250904 19:58:10.296509 13720 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:10.297384 10338 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20250904 19:58:10.297778 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/ts-1-root
uuid: "ab6ec379ac4a4790b8636bc0ab141e06"
format_stamp: "Formatted at 2025-09-04 19:58:10 on dist-test-slave-f3xj"
I20250904 19:58:10.298215 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:10.316728 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:10.318070 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:10.319782 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:10.323324 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:10.323585 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:10.323866 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:10.324074 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:10.398379 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.130:34573
I20250904 19:58:10.398634 13782 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.130:34573 every 8 connection(s)
I20250904 19:58:10.426214 13783 heartbeater.cc:344] Connected to a master server at 127.10.24.188:44757
I20250904 19:58:10.426689 13783 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:10.426920 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:10.427651 13783 heartbeater.cc:507] Master 127.10.24.188:44757 requested a full tablet report, sending...
I20250904 19:58:10.431622 13542 ts_manager.cc:194] Registered new tserver with Master: ab6ec379ac4a4790b8636bc0ab141e06 (127.10.24.130:34573)
I20250904 19:58:10.444842 13784 heartbeater.cc:344] Connected to a master server at 127.10.24.189:46123
I20250904 19:58:10.445259 13784 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:10.446146 13784 heartbeater.cc:507] Master 127.10.24.189:46123 requested a full tablet report, sending...
W20250904 19:58:10.446125 13790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:10.459709 13476 ts_manager.cc:194] Registered new tserver with Master: ab6ec379ac4a4790b8636bc0ab141e06 (127.10.24.130:34573)
I20250904 19:58:10.461992 13785 heartbeater.cc:344] Connected to a master server at 127.10.24.190:40239
I20250904 19:58:10.462903 13785 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:10.463747 13785 heartbeater.cc:507] Master 127.10.24.190:40239 requested a full tablet report, sending...
I20250904 19:58:10.465914 13411 ts_manager.cc:194] Registered new tserver with Master: ab6ec379ac4a4790b8636bc0ab141e06 (127.10.24.130:34573)
I20250904 19:58:10.468145 13411 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52394
W20250904 19:58:10.472323 13791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:10.472679 10338 server_base.cc:1047] running on GCE node
W20250904 19:58:10.476359 13794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:10.477573 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:10.477870 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:10.478096 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015890478078 us; error 0 us; skew 500 ppm
I20250904 19:58:10.478878 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:10.481215 13785 heartbeater.cc:499] Master 127.10.24.190:40239 was elected leader, sending a full tablet report...
I20250904 19:58:10.485155 10338 webserver.cc:480] Webserver started at http://127.10.24.131:38611/ using document root <none> and password file <none>
I20250904 19:58:10.485651 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:10.485818 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:10.486716 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:10.488010 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/ts-2-root/instance:
uuid: "d1674ffee63e4a2ba7debbd08af98942"
format_stamp: "Formatted at 2025-09-04 19:58:10 on dist-test-slave-f3xj"
I20250904 19:58:10.500056 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.011s user 0.002s sys 0.008s
I20250904 19:58:10.505323 13798 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:10.506152 10338 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.002s sys 0.003s
I20250904 19:58:10.506572 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/ts-2-root
uuid: "d1674ffee63e4a2ba7debbd08af98942"
format_stamp: "Formatted at 2025-09-04 19:58:10 on dist-test-slave-f3xj"
I20250904 19:58:10.506976 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumTimeout.1757015869512910-10338-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:10.525957 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:10.527467 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:10.529503 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:10.532621 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:10.532893 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:10.533238 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:10.533483 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:10.660416 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.131:35973
I20250904 19:58:10.660540 13860 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.131:35973 every 8 connection(s)
I20250904 19:58:10.714964 13861 heartbeater.cc:344] Connected to a master server at 127.10.24.188:44757
I20250904 19:58:10.715401 13861 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:10.716353 13861 heartbeater.cc:507] Master 127.10.24.188:44757 requested a full tablet report, sending...
I20250904 19:58:10.732858 13541 ts_manager.cc:194] Registered new tserver with Master: d1674ffee63e4a2ba7debbd08af98942 (127.10.24.131:35973)
I20250904 19:58:10.753479 13863 heartbeater.cc:344] Connected to a master server at 127.10.24.190:40239
I20250904 19:58:10.753943 13863 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:10.754863 13863 heartbeater.cc:507] Master 127.10.24.190:40239 requested a full tablet report, sending...
I20250904 19:58:10.759121 13410 ts_manager.cc:194] Registered new tserver with Master: d1674ffee63e4a2ba7debbd08af98942 (127.10.24.131:35973)
I20250904 19:58:10.759440 13862 heartbeater.cc:344] Connected to a master server at 127.10.24.189:46123
I20250904 19:58:10.759904 13862 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:10.760797 13862 heartbeater.cc:507] Master 127.10.24.189:46123 requested a full tablet report, sending...
I20250904 19:58:10.765944 13474 ts_manager.cc:194] Registered new tserver with Master: d1674ffee63e4a2ba7debbd08af98942 (127.10.24.131:35973)
I20250904 19:58:10.766758 10338 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.028641851s
I20250904 19:58:10.774544 13410 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52402
I20250904 19:58:10.789148 13863 heartbeater.cc:499] Master 127.10.24.190:40239 was elected leader, sending a full tablet report...
I20250904 19:58:10.925710 13410 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52404:
name: "ksck-test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\n\000\000\000\004\001\024\000\000\000""\004\001\n\000\000\000\004\001\024\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 19:58:10.928573 13410 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table ksck-test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 19:58:10.944628 13610 catalog_manager.cc:1261] Loaded cluster ID: 43a2c421c08b4d99b4d9382282ab8793
I20250904 19:58:10.950068 13610 catalog_manager.cc:1554] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5: loading cluster ID for follower catalog manager: success
I20250904 19:58:10.958518 13610 catalog_manager.cc:1576] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5: acquiring CA information for follower catalog manager: success
I20250904 19:58:10.965777 13610 catalog_manager.cc:1604] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:58:11.089926 13616 catalog_manager.cc:1261] Loaded cluster ID: 43a2c421c08b4d99b4d9382282ab8793
I20250904 19:58:11.090340 13616 catalog_manager.cc:1554] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388: loading cluster ID for follower catalog manager: success
I20250904 19:58:11.118856 13616 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388: acquiring CA information for follower catalog manager: success
I20250904 19:58:11.135321 13616 catalog_manager.cc:1604] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:58:11.191493 13748 tablet_service.cc:1468] Processing CreateTablet for tablet 4d35cb50e1a44fe19c7f168e4658586d (DEFAULT_TABLE table=ksck-test-table [id=64e802bf93a14a9ba602a4614a52b17e]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:11.193017 13748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d35cb50e1a44fe19c7f168e4658586d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:11.199448 13747 tablet_service.cc:1468] Processing CreateTablet for tablet 554fd682ad784eb7921a842bc8244e28 (DEFAULT_TABLE table=ksck-test-table [id=64e802bf93a14a9ba602a4614a52b17e]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:11.200932 13747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 554fd682ad784eb7921a842bc8244e28. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:11.203473 13746 tablet_service.cc:1468] Processing CreateTablet for tablet 1f59f407dece4ea1a6ebddeccc2b98ee (DEFAULT_TABLE table=ksck-test-table [id=64e802bf93a14a9ba602a4614a52b17e]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:11.204974 13746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f59f407dece4ea1a6ebddeccc2b98ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:11.215474 13826 tablet_service.cc:1468] Processing CreateTablet for tablet 4d35cb50e1a44fe19c7f168e4658586d (DEFAULT_TABLE table=ksck-test-table [id=64e802bf93a14a9ba602a4614a52b17e]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:11.216810 13826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d35cb50e1a44fe19c7f168e4658586d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:11.226492 13825 tablet_service.cc:1468] Processing CreateTablet for tablet 554fd682ad784eb7921a842bc8244e28 (DEFAULT_TABLE table=ksck-test-table [id=64e802bf93a14a9ba602a4614a52b17e]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:11.227869 13825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 554fd682ad784eb7921a842bc8244e28. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:11.238449 13670 tablet_service.cc:1468] Processing CreateTablet for tablet 4d35cb50e1a44fe19c7f168e4658586d (DEFAULT_TABLE table=ksck-test-table [id=64e802bf93a14a9ba602a4614a52b17e]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:11.239761 13670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d35cb50e1a44fe19c7f168e4658586d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:11.242591 13669 tablet_service.cc:1468] Processing CreateTablet for tablet 554fd682ad784eb7921a842bc8244e28 (DEFAULT_TABLE table=ksck-test-table [id=64e802bf93a14a9ba602a4614a52b17e]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:11.245618 13668 tablet_service.cc:1468] Processing CreateTablet for tablet 1f59f407dece4ea1a6ebddeccc2b98ee (DEFAULT_TABLE table=ksck-test-table [id=64e802bf93a14a9ba602a4614a52b17e]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:11.246953 13668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f59f407dece4ea1a6ebddeccc2b98ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:11.251910 13669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 554fd682ad784eb7921a842bc8244e28. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:11.252961 13824 tablet_service.cc:1468] Processing CreateTablet for tablet 1f59f407dece4ea1a6ebddeccc2b98ee (DEFAULT_TABLE table=ksck-test-table [id=64e802bf93a14a9ba602a4614a52b17e]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:11.254209 13824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f59f407dece4ea1a6ebddeccc2b98ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:11.283146 13890 tablet_bootstrap.cc:492] T 4d35cb50e1a44fe19c7f168e4658586d P ab6ec379ac4a4790b8636bc0ab141e06: Bootstrap starting.
I20250904 19:58:11.290113 13892 tablet_bootstrap.cc:492] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942: Bootstrap starting.
I20250904 19:58:11.303936 13890 tablet_bootstrap.cc:654] T 4d35cb50e1a44fe19c7f168e4658586d P ab6ec379ac4a4790b8636bc0ab141e06: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:11.315696 13891 tablet_bootstrap.cc:492] T 4d35cb50e1a44fe19c7f168e4658586d P 566e42931fac46a68901dc3725fe14d7: Bootstrap starting.
I20250904 19:58:11.322569 13891 tablet_bootstrap.cc:654] T 4d35cb50e1a44fe19c7f168e4658586d P 566e42931fac46a68901dc3725fe14d7: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:11.334882 13890 tablet_bootstrap.cc:492] T 4d35cb50e1a44fe19c7f168e4658586d P ab6ec379ac4a4790b8636bc0ab141e06: No bootstrap required, opened a new log
I20250904 19:58:11.335487 13890 ts_tablet_manager.cc:1397] T 4d35cb50e1a44fe19c7f168e4658586d P ab6ec379ac4a4790b8636bc0ab141e06: Time spent bootstrapping tablet: real 0.053s user 0.008s sys 0.003s
I20250904 19:58:11.337040 13892 tablet_bootstrap.cc:654] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:11.338702 13890 raft_consensus.cc:357] T 4d35cb50e1a44fe19c7f168e4658586d P ab6ec379ac4a4790b8636bc0ab141e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.349354 13890 raft_consensus.cc:383] T 4d35cb50e1a44fe19c7f168e4658586d P ab6ec379ac4a4790b8636bc0ab141e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:11.349733 13890 raft_consensus.cc:738] T 4d35cb50e1a44fe19c7f168e4658586d P ab6ec379ac4a4790b8636bc0ab141e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab6ec379ac4a4790b8636bc0ab141e06, State: Initialized, Role: FOLLOWER
I20250904 19:58:11.350608 13890 consensus_queue.cc:260] T 4d35cb50e1a44fe19c7f168e4658586d P ab6ec379ac4a4790b8636bc0ab141e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.363600 13890 ts_tablet_manager.cc:1428] T 4d35cb50e1a44fe19c7f168e4658586d P ab6ec379ac4a4790b8636bc0ab141e06: Time spent starting tablet: real 0.028s user 0.007s sys 0.004s
I20250904 19:58:11.364183 13892 tablet_bootstrap.cc:492] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942: No bootstrap required, opened a new log
I20250904 19:58:11.364719 13892 ts_tablet_manager.cc:1397] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942: Time spent bootstrapping tablet: real 0.075s user 0.013s sys 0.000s
I20250904 19:58:11.368048 13892 raft_consensus.cc:357] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.370060 13892 raft_consensus.cc:383] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:11.370429 13892 raft_consensus.cc:738] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1674ffee63e4a2ba7debbd08af98942, State: Initialized, Role: FOLLOWER
I20250904 19:58:11.371237 13892 consensus_queue.cc:260] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.374035 13891 tablet_bootstrap.cc:492] T 4d35cb50e1a44fe19c7f168e4658586d P 566e42931fac46a68901dc3725fe14d7: No bootstrap required, opened a new log
I20250904 19:58:11.374538 13891 ts_tablet_manager.cc:1397] T 4d35cb50e1a44fe19c7f168e4658586d P 566e42931fac46a68901dc3725fe14d7: Time spent bootstrapping tablet: real 0.059s user 0.017s sys 0.009s
I20250904 19:58:11.380913 13890 tablet_bootstrap.cc:492] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06: Bootstrap starting.
I20250904 19:58:11.382579 13891 raft_consensus.cc:357] T 4d35cb50e1a44fe19c7f168e4658586d P 566e42931fac46a68901dc3725fe14d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.395094 13890 tablet_bootstrap.cc:654] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:11.404457 13891 raft_consensus.cc:383] T 4d35cb50e1a44fe19c7f168e4658586d P 566e42931fac46a68901dc3725fe14d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:11.404820 13891 raft_consensus.cc:738] T 4d35cb50e1a44fe19c7f168e4658586d P 566e42931fac46a68901dc3725fe14d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 566e42931fac46a68901dc3725fe14d7, State: Initialized, Role: FOLLOWER
I20250904 19:58:11.406936 13891 consensus_queue.cc:260] T 4d35cb50e1a44fe19c7f168e4658586d P 566e42931fac46a68901dc3725fe14d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.416225 13892 ts_tablet_manager.cc:1428] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942: Time spent starting tablet: real 0.051s user 0.006s sys 0.012s
I20250904 19:58:11.420291 13892 tablet_bootstrap.cc:492] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942: Bootstrap starting.
I20250904 19:58:11.445683 13890 tablet_bootstrap.cc:492] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06: No bootstrap required, opened a new log
I20250904 19:58:11.446316 13890 ts_tablet_manager.cc:1397] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06: Time spent bootstrapping tablet: real 0.066s user 0.014s sys 0.001s
I20250904 19:58:11.449262 13890 raft_consensus.cc:357] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.450032 13890 raft_consensus.cc:383] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:11.450397 13890 raft_consensus.cc:738] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab6ec379ac4a4790b8636bc0ab141e06, State: Initialized, Role: FOLLOWER
I20250904 19:58:11.450523 13892 tablet_bootstrap.cc:654] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:11.451349 13890 consensus_queue.cc:260] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.477535 13892 tablet_bootstrap.cc:492] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942: No bootstrap required, opened a new log
I20250904 19:58:11.477588 13890 ts_tablet_manager.cc:1428] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06: Time spent starting tablet: real 0.031s user 0.006s sys 0.000s
I20250904 19:58:11.478406 13892 ts_tablet_manager.cc:1397] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942: Time spent bootstrapping tablet: real 0.058s user 0.009s sys 0.011s
I20250904 19:58:11.479336 13890 tablet_bootstrap.cc:492] T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06: Bootstrap starting.
I20250904 19:58:11.488921 13892 raft_consensus.cc:357] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.489753 13892 raft_consensus.cc:383] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:11.490538 13892 raft_consensus.cc:738] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1674ffee63e4a2ba7debbd08af98942, State: Initialized, Role: FOLLOWER
I20250904 19:58:11.500680 13891 ts_tablet_manager.cc:1428] T 4d35cb50e1a44fe19c7f168e4658586d P 566e42931fac46a68901dc3725fe14d7: Time spent starting tablet: real 0.126s user 0.008s sys 0.000s
I20250904 19:58:11.494069 13892 consensus_queue.cc:260] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.505874 13891 tablet_bootstrap.cc:492] T 1f59f407dece4ea1a6ebddeccc2b98ee P 566e42931fac46a68901dc3725fe14d7: Bootstrap starting.
I20250904 19:58:11.518486 13892 ts_tablet_manager.cc:1428] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942: Time spent starting tablet: real 0.028s user 0.003s sys 0.003s
I20250904 19:58:11.521034 13892 tablet_bootstrap.cc:492] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942: Bootstrap starting.
I20250904 19:58:11.530615 13890 tablet_bootstrap.cc:654] T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:11.548466 13891 tablet_bootstrap.cc:654] T 1f59f407dece4ea1a6ebddeccc2b98ee P 566e42931fac46a68901dc3725fe14d7: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:11.560163 13890 tablet_bootstrap.cc:492] T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06: No bootstrap required, opened a new log
I20250904 19:58:11.560802 13890 ts_tablet_manager.cc:1397] T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06: Time spent bootstrapping tablet: real 0.082s user 0.019s sys 0.001s
I20250904 19:58:11.562866 13892 tablet_bootstrap.cc:654] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:11.564920 13890 raft_consensus.cc:357] T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.565822 13890 raft_consensus.cc:383] T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:11.568212 13892 tablet_bootstrap.cc:492] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942: No bootstrap required, opened a new log
I20250904 19:58:11.568725 13892 ts_tablet_manager.cc:1397] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942: Time spent bootstrapping tablet: real 0.048s user 0.008s sys 0.004s
I20250904 19:58:11.566179 13890 raft_consensus.cc:738] T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab6ec379ac4a4790b8636bc0ab141e06, State: Initialized, Role: FOLLOWER
I20250904 19:58:11.572263 13890 consensus_queue.cc:260] T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.574936 13890 ts_tablet_manager.cc:1428] T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06: Time spent starting tablet: real 0.014s user 0.006s sys 0.001s
I20250904 19:58:11.584146 13891 tablet_bootstrap.cc:492] T 1f59f407dece4ea1a6ebddeccc2b98ee P 566e42931fac46a68901dc3725fe14d7: No bootstrap required, opened a new log
I20250904 19:58:11.585009 13891 ts_tablet_manager.cc:1397] T 1f59f407dece4ea1a6ebddeccc2b98ee P 566e42931fac46a68901dc3725fe14d7: Time spent bootstrapping tablet: real 0.079s user 0.012s sys 0.000s
I20250904 19:58:11.584057 13892 raft_consensus.cc:357] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.588238 13892 raft_consensus.cc:383] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:11.588521 13892 raft_consensus.cc:738] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1674ffee63e4a2ba7debbd08af98942, State: Initialized, Role: FOLLOWER
I20250904 19:58:11.589226 13892 consensus_queue.cc:260] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.588907 13891 raft_consensus.cc:357] T 1f59f407dece4ea1a6ebddeccc2b98ee P 566e42931fac46a68901dc3725fe14d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.590065 13891 raft_consensus.cc:383] T 1f59f407dece4ea1a6ebddeccc2b98ee P 566e42931fac46a68901dc3725fe14d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:11.590400 13891 raft_consensus.cc:738] T 1f59f407dece4ea1a6ebddeccc2b98ee P 566e42931fac46a68901dc3725fe14d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 566e42931fac46a68901dc3725fe14d7, State: Initialized, Role: FOLLOWER
I20250904 19:58:11.591522 13892 ts_tablet_manager.cc:1428] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942: Time spent starting tablet: real 0.022s user 0.002s sys 0.001s
I20250904 19:58:11.591216 13891 consensus_queue.cc:260] T 1f59f407dece4ea1a6ebddeccc2b98ee P 566e42931fac46a68901dc3725fe14d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.601938 13891 ts_tablet_manager.cc:1428] T 1f59f407dece4ea1a6ebddeccc2b98ee P 566e42931fac46a68901dc3725fe14d7: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20250904 19:58:11.602870 13895 raft_consensus.cc:491] T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:11.603420 13891 tablet_bootstrap.cc:492] T 554fd682ad784eb7921a842bc8244e28 P 566e42931fac46a68901dc3725fe14d7: Bootstrap starting.
I20250904 19:58:11.603371 13895 raft_consensus.cc:513] T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.605896 13895 leader_election.cc:290] T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 566e42931fac46a68901dc3725fe14d7 (127.10.24.129:44369), d1674ffee63e4a2ba7debbd08af98942 (127.10.24.131:35973)
I20250904 19:58:11.606483 13897 raft_consensus.cc:491] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:11.606967 13897 raft_consensus.cc:513] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.610009 13891 tablet_bootstrap.cc:654] T 554fd682ad784eb7921a842bc8244e28 P 566e42931fac46a68901dc3725fe14d7: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:11.617805 13897 leader_election.cc:290] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 566e42931fac46a68901dc3725fe14d7 (127.10.24.129:44369), ab6ec379ac4a4790b8636bc0ab141e06 (127.10.24.130:34573)
I20250904 19:58:11.660514 13891 tablet_bootstrap.cc:492] T 554fd682ad784eb7921a842bc8244e28 P 566e42931fac46a68901dc3725fe14d7: No bootstrap required, opened a new log
I20250904 19:58:11.661152 13891 ts_tablet_manager.cc:1397] T 554fd682ad784eb7921a842bc8244e28 P 566e42931fac46a68901dc3725fe14d7: Time spent bootstrapping tablet: real 0.058s user 0.019s sys 0.007s
I20250904 19:58:11.664054 13891 raft_consensus.cc:357] T 554fd682ad784eb7921a842bc8244e28 P 566e42931fac46a68901dc3725fe14d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.669564 13891 raft_consensus.cc:383] T 554fd682ad784eb7921a842bc8244e28 P 566e42931fac46a68901dc3725fe14d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:11.669941 13891 raft_consensus.cc:738] T 554fd682ad784eb7921a842bc8244e28 P 566e42931fac46a68901dc3725fe14d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 566e42931fac46a68901dc3725fe14d7, State: Initialized, Role: FOLLOWER
I20250904 19:58:11.674865 13891 consensus_queue.cc:260] T 554fd682ad784eb7921a842bc8244e28 P 566e42931fac46a68901dc3725fe14d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.684206 13891 ts_tablet_manager.cc:1428] T 554fd682ad784eb7921a842bc8244e28 P 566e42931fac46a68901dc3725fe14d7: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
I20250904 19:58:11.700819 13680 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1f59f407dece4ea1a6ebddeccc2b98ee" candidate_uuid: "d1674ffee63e4a2ba7debbd08af98942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "566e42931fac46a68901dc3725fe14d7" is_pre_election: true
I20250904 19:58:11.701534 13680 raft_consensus.cc:2466] T 1f59f407dece4ea1a6ebddeccc2b98ee P 566e42931fac46a68901dc3725fe14d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1674ffee63e4a2ba7debbd08af98942 in term 0.
I20250904 19:58:11.702924 13801 leader_election.cc:304] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 566e42931fac46a68901dc3725fe14d7, d1674ffee63e4a2ba7debbd08af98942; no voters:
I20250904 19:58:11.703966 13897 raft_consensus.cc:2802] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:11.704391 13897 raft_consensus.cc:491] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:11.704715 13897 raft_consensus.cc:3058] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:11.713050 13895 raft_consensus.cc:491] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:11.713574 13895 raft_consensus.cc:513] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.716737 13895 leader_election.cc:290] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 566e42931fac46a68901dc3725fe14d7 (127.10.24.129:44369), d1674ffee63e4a2ba7debbd08af98942 (127.10.24.131:35973)
I20250904 19:58:11.716931 13836 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1f59f407dece4ea1a6ebddeccc2b98ee" candidate_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1674ffee63e4a2ba7debbd08af98942" is_pre_election: true
I20250904 19:58:11.720682 13835 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "554fd682ad784eb7921a842bc8244e28" candidate_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1674ffee63e4a2ba7debbd08af98942" is_pre_election: true
I20250904 19:58:11.721313 13835 raft_consensus.cc:2466] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab6ec379ac4a4790b8636bc0ab141e06 in term 0.
I20250904 19:58:11.724429 13722 leader_election.cc:304] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab6ec379ac4a4790b8636bc0ab141e06, d1674ffee63e4a2ba7debbd08af98942; no voters:
I20250904 19:58:11.727779 13895 raft_consensus.cc:2802] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:11.728199 13895 raft_consensus.cc:491] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:11.728585 13895 raft_consensus.cc:3058] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:11.728813 13897 raft_consensus.cc:513] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.731333 13836 raft_consensus.cc:2391] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ab6ec379ac4a4790b8636bc0ab141e06 in current term 1: Already voted for candidate d1674ffee63e4a2ba7debbd08af98942 in this term.
I20250904 19:58:11.732170 13910 raft_consensus.cc:491] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:11.732654 13910 raft_consensus.cc:513] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.735504 13680 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "554fd682ad784eb7921a842bc8244e28" candidate_uuid: "d1674ffee63e4a2ba7debbd08af98942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "566e42931fac46a68901dc3725fe14d7" is_pre_election: true
I20250904 19:58:11.736142 13680 raft_consensus.cc:2466] T 554fd682ad784eb7921a842bc8244e28 P 566e42931fac46a68901dc3725fe14d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1674ffee63e4a2ba7debbd08af98942 in term 0.
I20250904 19:58:11.737216 13801 leader_election.cc:304] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 566e42931fac46a68901dc3725fe14d7, d1674ffee63e4a2ba7debbd08af98942; no voters:
I20250904 19:58:11.742448 13912 raft_consensus.cc:2802] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:11.743544 13912 raft_consensus.cc:491] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:11.743988 13912 raft_consensus.cc:3058] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:11.737871 13895 raft_consensus.cc:513] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.753120 13895 leader_election.cc:290] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [CANDIDATE]: Term 1 election: Requested vote from peers 566e42931fac46a68901dc3725fe14d7 (127.10.24.129:44369), d1674ffee63e4a2ba7debbd08af98942 (127.10.24.131:35973)
I20250904 19:58:11.753054 13758 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1f59f407dece4ea1a6ebddeccc2b98ee" candidate_uuid: "d1674ffee63e4a2ba7debbd08af98942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" is_pre_election: true
I20250904 19:58:11.750818 13910 leader_election.cc:290] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 566e42931fac46a68901dc3725fe14d7 (127.10.24.129:44369), ab6ec379ac4a4790b8636bc0ab141e06 (127.10.24.130:34573)
I20250904 19:58:11.753897 13910 raft_consensus.cc:491] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:11.754489 13836 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "554fd682ad784eb7921a842bc8244e28" candidate_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1674ffee63e4a2ba7debbd08af98942"
I20250904 19:58:11.754431 13910 raft_consensus.cc:513] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.753860 13758 raft_consensus.cc:2466] T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1674ffee63e4a2ba7debbd08af98942 in term 0.
I20250904 19:58:11.759871 13912 raft_consensus.cc:513] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.761092 13836 raft_consensus.cc:2391] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ab6ec379ac4a4790b8636bc0ab141e06 in current term 1: Already voted for candidate d1674ffee63e4a2ba7debbd08af98942 in this term.
I20250904 19:58:11.764258 13758 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "554fd682ad784eb7921a842bc8244e28" candidate_uuid: "d1674ffee63e4a2ba7debbd08af98942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" is_pre_election: true
I20250904 19:58:11.764463 13756 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4d35cb50e1a44fe19c7f168e4658586d" candidate_uuid: "d1674ffee63e4a2ba7debbd08af98942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" is_pre_election: true
I20250904 19:58:11.757503 13910 leader_election.cc:290] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 566e42931fac46a68901dc3725fe14d7 (127.10.24.129:44369), ab6ec379ac4a4790b8636bc0ab141e06 (127.10.24.130:34573)
I20250904 19:58:11.765022 13758 raft_consensus.cc:2391] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1674ffee63e4a2ba7debbd08af98942 in current term 1: Already voted for candidate ab6ec379ac4a4790b8636bc0ab141e06 in this term.
I20250904 19:58:11.765031 13756 raft_consensus.cc:2466] T 4d35cb50e1a44fe19c7f168e4658586d P ab6ec379ac4a4790b8636bc0ab141e06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1674ffee63e4a2ba7debbd08af98942 in term 0.
I20250904 19:58:11.770028 13912 leader_election.cc:290] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942 [CANDIDATE]: Term 1 election: Requested vote from peers 566e42931fac46a68901dc3725fe14d7 (127.10.24.129:44369), ab6ec379ac4a4790b8636bc0ab141e06 (127.10.24.130:34573)
I20250904 19:58:11.771168 13678 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "554fd682ad784eb7921a842bc8244e28" candidate_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "566e42931fac46a68901dc3725fe14d7"
I20250904 19:58:11.773541 13678 raft_consensus.cc:3058] T 554fd682ad784eb7921a842bc8244e28 P 566e42931fac46a68901dc3725fe14d7 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:11.771368 13679 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "554fd682ad784eb7921a842bc8244e28" candidate_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "566e42931fac46a68901dc3725fe14d7" is_pre_election: true
I20250904 19:58:11.771770 13680 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1f59f407dece4ea1a6ebddeccc2b98ee" candidate_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "566e42931fac46a68901dc3725fe14d7" is_pre_election: true
I20250904 19:58:11.776540 13680 raft_consensus.cc:2466] T 1f59f407dece4ea1a6ebddeccc2b98ee P 566e42931fac46a68901dc3725fe14d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab6ec379ac4a4790b8636bc0ab141e06 in term 0.
I20250904 19:58:11.778542 13723 leader_election.cc:304] T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 566e42931fac46a68901dc3725fe14d7, ab6ec379ac4a4790b8636bc0ab141e06; no voters: d1674ffee63e4a2ba7debbd08af98942
I20250904 19:58:11.782565 13756 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1f59f407dece4ea1a6ebddeccc2b98ee" candidate_uuid: "d1674ffee63e4a2ba7debbd08af98942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab6ec379ac4a4790b8636bc0ab141e06"
I20250904 19:58:11.785868 13799 leader_election.cc:304] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab6ec379ac4a4790b8636bc0ab141e06, d1674ffee63e4a2ba7debbd08af98942; no voters:
I20250904 19:58:11.786005 13756 raft_consensus.cc:3058] T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:11.772359 13897 leader_election.cc:290] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [CANDIDATE]: Term 1 election: Requested vote from peers 566e42931fac46a68901dc3725fe14d7 (127.10.24.129:44369), ab6ec379ac4a4790b8636bc0ab141e06 (127.10.24.130:34573)
I20250904 19:58:11.789561 13678 raft_consensus.cc:2466] T 554fd682ad784eb7921a842bc8244e28 P 566e42931fac46a68901dc3725fe14d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab6ec379ac4a4790b8636bc0ab141e06 in term 1.
I20250904 19:58:11.790701 13723 leader_election.cc:304] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 566e42931fac46a68901dc3725fe14d7, ab6ec379ac4a4790b8636bc0ab141e06; no voters: d1674ffee63e4a2ba7debbd08af98942
W20250904 19:58:11.791229 13801 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.24.129:44369, user_credentials={real_user=slave}} blocked reactor thread for 54116.4us
I20250904 19:58:11.793512 13678 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4d35cb50e1a44fe19c7f168e4658586d" candidate_uuid: "d1674ffee63e4a2ba7debbd08af98942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "566e42931fac46a68901dc3725fe14d7" is_pre_election: true
I20250904 19:58:11.794167 13678 raft_consensus.cc:2466] T 4d35cb50e1a44fe19c7f168e4658586d P 566e42931fac46a68901dc3725fe14d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1674ffee63e4a2ba7debbd08af98942 in term 0.
I20250904 19:58:11.796844 13680 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "554fd682ad784eb7921a842bc8244e28" candidate_uuid: "d1674ffee63e4a2ba7debbd08af98942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "566e42931fac46a68901dc3725fe14d7"
I20250904 19:58:11.797569 13680 raft_consensus.cc:2391] T 554fd682ad784eb7921a842bc8244e28 P 566e42931fac46a68901dc3725fe14d7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d1674ffee63e4a2ba7debbd08af98942 in current term 1: Already voted for candidate ab6ec379ac4a4790b8636bc0ab141e06 in this term.
I20250904 19:58:11.785058 13758 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "554fd682ad784eb7921a842bc8244e28" candidate_uuid: "d1674ffee63e4a2ba7debbd08af98942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab6ec379ac4a4790b8636bc0ab141e06"
I20250904 19:58:11.798635 13758 raft_consensus.cc:2391] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d1674ffee63e4a2ba7debbd08af98942 in current term 1: Already voted for candidate ab6ec379ac4a4790b8636bc0ab141e06 in this term.
I20250904 19:58:11.798549 13679 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1f59f407dece4ea1a6ebddeccc2b98ee" candidate_uuid: "d1674ffee63e4a2ba7debbd08af98942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "566e42931fac46a68901dc3725fe14d7"
I20250904 19:58:11.802280 13679 raft_consensus.cc:3058] T 1f59f407dece4ea1a6ebddeccc2b98ee P 566e42931fac46a68901dc3725fe14d7 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:11.802814 13756 raft_consensus.cc:2466] T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1674ffee63e4a2ba7debbd08af98942 in term 1.
I20250904 19:58:11.804795 13895 raft_consensus.cc:2762] T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20250904 19:58:11.807400 13895 raft_consensus.cc:2802] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:11.811905 13895 raft_consensus.cc:695] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [term 1 LEADER]: Becoming Leader. State: Replica: ab6ec379ac4a4790b8636bc0ab141e06, State: Running, Role: LEADER
I20250904 19:58:11.812675 13895 consensus_queue.cc:237] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.821079 13679 raft_consensus.cc:2466] T 1f59f407dece4ea1a6ebddeccc2b98ee P 566e42931fac46a68901dc3725fe14d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1674ffee63e4a2ba7debbd08af98942 in term 1.
I20250904 19:58:11.827903 13801 leader_election.cc:304] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 566e42931fac46a68901dc3725fe14d7, d1674ffee63e4a2ba7debbd08af98942; no voters:
I20250904 19:58:11.828799 13897 raft_consensus.cc:2802] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:11.829407 13897 raft_consensus.cc:695] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [term 1 LEADER]: Becoming Leader. State: Replica: d1674ffee63e4a2ba7debbd08af98942, State: Running, Role: LEADER
I20250904 19:58:11.830209 13897 consensus_queue.cc:237] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
W20250904 19:58:11.831101 13799 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.24.130:34573, user_credentials={real_user=slave}} blocked reactor thread for 45447.2us
I20250904 19:58:11.830471 13912 raft_consensus.cc:2802] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:11.832306 13912 raft_consensus.cc:491] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:11.832703 13912 raft_consensus.cc:3058] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:11.832489 13799 leader_election.cc:304] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d1674ffee63e4a2ba7debbd08af98942; no voters: 566e42931fac46a68901dc3725fe14d7, ab6ec379ac4a4790b8636bc0ab141e06
I20250904 19:58:11.835875 13911 raft_consensus.cc:2747] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250904 19:58:11.844842 13410 catalog_manager.cc:5582] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 reported cstate change: term changed from 0 to 1, leader changed from <none> to ab6ec379ac4a4790b8636bc0ab141e06 (127.10.24.130). New cstate: current_term: 1 leader_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } health_report { overall_health: UNKNOWN } } }
I20250904 19:58:11.848778 13912 raft_consensus.cc:513] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.859571 13912 leader_election.cc:290] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [CANDIDATE]: Term 1 election: Requested vote from peers 566e42931fac46a68901dc3725fe14d7 (127.10.24.129:44369), ab6ec379ac4a4790b8636bc0ab141e06 (127.10.24.130:34573)
I20250904 19:58:11.860278 13679 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4d35cb50e1a44fe19c7f168e4658586d" candidate_uuid: "d1674ffee63e4a2ba7debbd08af98942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "566e42931fac46a68901dc3725fe14d7"
I20250904 19:58:11.860941 13679 raft_consensus.cc:3058] T 4d35cb50e1a44fe19c7f168e4658586d P 566e42931fac46a68901dc3725fe14d7 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:11.865816 13408 catalog_manager.cc:5582] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1674ffee63e4a2ba7debbd08af98942 (127.10.24.131). New cstate: current_term: 1 leader_uuid: "d1674ffee63e4a2ba7debbd08af98942" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } health_report { overall_health: HEALTHY } } }
I20250904 19:58:11.870018 13679 raft_consensus.cc:2466] T 4d35cb50e1a44fe19c7f168e4658586d P 566e42931fac46a68901dc3725fe14d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1674ffee63e4a2ba7debbd08af98942 in term 1.
I20250904 19:58:11.870971 13801 leader_election.cc:304] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 566e42931fac46a68901dc3725fe14d7, d1674ffee63e4a2ba7debbd08af98942; no voters:
I20250904 19:58:11.872611 13912 raft_consensus.cc:2802] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:11.873140 13912 raft_consensus.cc:695] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [term 1 LEADER]: Becoming Leader. State: Replica: d1674ffee63e4a2ba7debbd08af98942, State: Running, Role: LEADER
I20250904 19:58:11.873960 13912 consensus_queue.cc:237] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } }
I20250904 19:58:11.873863 13756 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4d35cb50e1a44fe19c7f168e4658586d" candidate_uuid: "d1674ffee63e4a2ba7debbd08af98942" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab6ec379ac4a4790b8636bc0ab141e06"
I20250904 19:58:11.876114 13756 raft_consensus.cc:3058] T 4d35cb50e1a44fe19c7f168e4658586d P ab6ec379ac4a4790b8636bc0ab141e06 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:11.886703 13756 raft_consensus.cc:2466] T 4d35cb50e1a44fe19c7f168e4658586d P ab6ec379ac4a4790b8636bc0ab141e06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1674ffee63e4a2ba7debbd08af98942 in term 1.
I20250904 19:58:11.962364 13408 catalog_manager.cc:5582] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 reported cstate change: term changed from 0 to 1, leader changed from <none> to d1674ffee63e4a2ba7debbd08af98942 (127.10.24.131). New cstate: current_term: 1 leader_uuid: "d1674ffee63e4a2ba7debbd08af98942" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 } health_report { overall_health: HEALTHY } } }
I20250904 19:58:12.061997 10338 ksck_remote-test.cc:410] Generating row writes...
I20250904 19:58:12.205761 13913 consensus_queue.cc:1035] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1674ffee63e4a2ba7debbd08af98942" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 35973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:12.236883 13912 consensus_queue.cc:1035] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:12.248957 13913 consensus_queue.cc:1035] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:12.297538 13911 consensus_queue.cc:1035] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [LEADER]: Connected to new peer: Peer: permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:12.320132 13911 consensus_queue.cc:1035] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [LEADER]: Connected to new peer: Peer: permanent_uuid: "566e42931fac46a68901dc3725fe14d7" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 44369 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:12.336802 13911 consensus_queue.cc:1035] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab6ec379ac4a4790b8636bc0ab141e06" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 34573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250904 19:58:14.004967 13969 log.cc:927] Time spent T 1f59f407dece4ea1a6ebddeccc2b98ee P 566e42931fac46a68901dc3725fe14d7: Append to log took a long time: real 0.106s user 0.010s sys 0.004s
W20250904 19:58:14.033293 13968 log.cc:927] Time spent T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06: Append to log took a long time: real 0.097s user 0.013s sys 0.000s
W20250904 19:58:15.006726 13969 log.cc:927] Time spent T 1f59f407dece4ea1a6ebddeccc2b98ee P 566e42931fac46a68901dc3725fe14d7: Append to log took a long time: real 0.059s user 0.013s sys 0.000s
I20250904 19:58:15.499022 13970 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:49886 (ReqId={client: 8f59c0b1d2b243f69631b67512dc0b50, seq_no=1, attempt_no=0}) took 2129 ms. Trace:
I20250904 19:58:15.499513 13970 rpcz_store.cc:276] 0904 19:58:13.369162 (+ 0us) service_pool.cc:168] Inserting onto call queue
0904 19:58:13.369321 (+ 159us) service_pool.cc:225] Handling call
0904 19:58:15.498956 (+2129635us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0904 19:58:13.389655 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 1f59f407dece4ea1a6ebddeccc2b98ee
0904 19:58:13.391622 (+ 1967us) write_op.cc:432] Acquiring schema lock in shared mode
0904 19:58:13.391651 (+ 29us) write_op.cc:435] Acquired schema lock
0904 19:58:13.391667 (+ 16us) tablet.cc:660] Decoding operations
0904 19:58:13.661651 (+269984us) write_op.cc:620] Acquiring the partition lock for write op
0904 19:58:13.661714 (+ 63us) write_op.cc:641] Partition lock acquired for write op
0904 19:58:13.661736 (+ 22us) tablet.cc:683] Acquiring locks for 3854 operations
0904 19:58:13.791016 (+129280us) tablet.cc:699] Row locks acquired
0904 19:58:13.791032 (+ 16us) write_op.cc:260] PREPARE: finished
0904 19:58:13.791193 (+ 161us) write_op.cc:270] Start()
0904 19:58:13.791301 (+ 108us) write_op.cc:276] Timestamp: P: 1757015893791153 usec, L: 0
0904 19:58:13.791319 (+ 18us) op_driver.cc:348] REPLICATION: starting
0904 19:58:13.792106 (+ 787us) log.cc:844] Serialized 38827 byte log entry
0904 19:58:14.012729 (+220623us) op_driver.cc:464] REPLICATION: finished
0904 19:58:14.077451 (+ 64722us) write_op.cc:301] APPLY: starting
0904 19:58:14.077532 (+ 81us) tablet.cc:1365] starting BulkCheckPresence
0904 19:58:14.371506 (+293974us) tablet.cc:1368] finished BulkCheckPresence
0904 19:58:14.371613 (+ 107us) tablet.cc:1370] starting ApplyRowOperation cycle
0904 19:58:15.311529 (+939916us) tablet.cc:1381] finished ApplyRowOperation cycle
0904 19:58:15.393030 (+ 81501us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0904 19:58:15.393083 (+ 53us) write_op.cc:312] APPLY: finished
0904 19:58:15.447906 (+ 54823us) log.cc:844] Serialized 23146 byte log entry
0904 19:58:15.448118 (+ 212us) write_op.cc:489] Releasing partition, row and schema locks
0904 19:58:15.494018 (+ 45900us) write_op.cc:454] Released schema lock
0904 19:58:15.496591 (+ 2573us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"spinlock_wait_cycles":45952,"child_traces":[["op",{"apply.queue_time_us":64515,"num_ops":3854,"prepare.queue_time_us":18804,"prepare.run_cpu_time_us":221620,"prepare.run_wall_time_us":421042,"raft.queue_time_us":3325,"raft.run_cpu_time_us":2212,"raft.run_wall_time_us":2212,"replication_time_us":221291,"spinlock_wait_cycles":18816,"thread_start_us":106394,"threads_started":3,"wal-append.queue_time_us":17596,"wal-append.run_cpu_time_us":13086,"wal-append.run_wall_time_us":1016428}]]}
W20250904 19:58:15.672461 13870 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.10.24.131:35973, user_credentials={real_user=slave}} blocked reactor thread for 89072.2us
I20250904 19:58:16.309119 13978 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:49886 (ReqId={client: 8f59c0b1d2b243f69631b67512dc0b50, seq_no=3, attempt_no=0}) took 1880 ms. Trace:
I20250904 19:58:16.309638 13978 rpcz_store.cc:276] 0904 19:58:14.428609 (+ 0us) service_pool.cc:168] Inserting onto call queue
0904 19:58:14.428754 (+ 145us) service_pool.cc:225] Handling call
0904 19:58:16.309059 (+1880305us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0904 19:58:14.446824 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 1f59f407dece4ea1a6ebddeccc2b98ee
0904 19:58:14.447230 (+ 406us) write_op.cc:432] Acquiring schema lock in shared mode
0904 19:58:14.447263 (+ 33us) write_op.cc:435] Acquired schema lock
0904 19:58:14.447280 (+ 17us) tablet.cc:660] Decoding operations
0904 19:58:14.743921 (+296641us) write_op.cc:620] Acquiring the partition lock for write op
0904 19:58:14.743972 (+ 51us) write_op.cc:641] Partition lock acquired for write op
0904 19:58:14.743995 (+ 23us) tablet.cc:683] Acquiring locks for 3825 operations
0904 19:58:14.859704 (+115709us) tablet.cc:699] Row locks acquired
0904 19:58:14.859721 (+ 17us) write_op.cc:260] PREPARE: finished
0904 19:58:14.859890 (+ 169us) write_op.cc:270] Start()
0904 19:58:14.859984 (+ 94us) write_op.cc:276] Timestamp: P: 1757015894859849 usec, L: 0
0904 19:58:14.860002 (+ 18us) op_driver.cc:348] REPLICATION: starting
0904 19:58:14.860738 (+ 736us) log.cc:844] Serialized 38546 byte log entry
0904 19:58:14.924384 (+ 63646us) op_driver.cc:464] REPLICATION: finished
0904 19:58:14.947459 (+ 23075us) write_op.cc:301] APPLY: starting
0904 19:58:14.947535 (+ 76us) tablet.cc:1365] starting BulkCheckPresence
0904 19:58:15.400257 (+452722us) tablet.cc:1368] finished BulkCheckPresence
0904 19:58:15.400362 (+ 105us) tablet.cc:1370] starting ApplyRowOperation cycle
0904 19:58:16.197738 (+797376us) tablet.cc:1381] finished ApplyRowOperation cycle
0904 19:58:16.227739 (+ 30001us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0904 19:58:16.227779 (+ 40us) write_op.cc:312] APPLY: finished
0904 19:58:16.277252 (+ 49473us) log.cc:844] Serialized 22972 byte log entry
0904 19:58:16.277442 (+ 190us) write_op.cc:489] Releasing partition, row and schema locks
0904 19:58:16.305336 (+ 27894us) write_op.cc:454] Released schema lock
0904 19:58:16.307517 (+ 2181us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":22857,"num_ops":3825,"prepare.queue_time_us":16175,"prepare.run_cpu_time_us":211120,"prepare.run_wall_time_us":426731,"raft.queue_time_us":256,"raft.run_cpu_time_us":1715,"raft.run_wall_time_us":1716,"replication_time_us":64259,"spinlock_wait_cycles":83072,"thread_start_us":81148,"threads_started":3,"wal-append.queue_time_us":884}]]}
W20250904 19:58:16.409039 13870 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.10.24.131:35973, user_credentials={real_user=slave}} blocked reactor thread for 91293.5us
W20250904 19:58:16.506186 13857 debug-util.cc:398] Leaking SignalData structure 0x7b08000d8620 after lost signal to thread 10341
W20250904 19:58:16.507294 13857 debug-util.cc:398] Leaking SignalData structure 0x7b080014e820 after lost signal to thread 13445
W20250904 19:58:16.509588 13857 debug-util.cc:398] Leaking SignalData structure 0x7b08000faa00 after lost signal to thread 13510
W20250904 19:58:16.510531 13857 debug-util.cc:398] Leaking SignalData structure 0x7b08001b75c0 after lost signal to thread 13577
W20250904 19:58:16.512141 13857 debug-util.cc:398] Leaking SignalData structure 0x7b08001d2ac0 after lost signal to thread 13704
W20250904 19:58:16.513096 13857 debug-util.cc:398] Leaking SignalData structure 0x7b08001a0800 after lost signal to thread 13782
W20250904 19:58:16.513931 13857 debug-util.cc:398] Leaking SignalData structure 0x7b0800121100 after lost signal to thread 13860
W20250904 19:58:17.213250 13870 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.10.24.131:35973, user_credentials={real_user=slave}} blocked reactor thread for 60709.3us
I20250904 19:58:17.818888 13746 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250904 19:58:17.821915 13670 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250904 19:58:17.854161 13824 tablet_service.cc:1430] Tablet server has 2 leaders and 0 scanners
I20250904 19:58:18.341463 10338 tablet_server.cc:178] TabletServer@127.10.24.129:0 shutting down...
W20250904 19:58:18.345503 13928 ksck_checksum.cc:286] failed to submit task to start additional tablet checksums: Service unavailable: The pool has been shut down.
W20250904 19:58:18.346556 13928 ksck_checksum.cc:286] failed to submit task to start additional tablet checksums: Service unavailable: The pool has been shut down.
W20250904 19:58:18.356431 13801 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 14 similar messages]
W20250904 19:58:18.360019 13927 ksck_checksum.cc:286] failed to submit task to start additional tablet checksums: Service unavailable: The pool has been shut down.
W20250904 19:58:18.362571 13926 ksck_checksum.cc:286] failed to submit task to start additional tablet checksums: Service unavailable: The pool has been shut down.
W20250904 19:58:18.365523 13928 ksck_checksum.cc:286] failed to submit task to start additional tablet checksums: Service unavailable: The pool has been shut down.
W20250904 19:58:18.366264 13926 ksck_checksum.cc:286] failed to submit task to start additional tablet checksums: Service unavailable: The pool has been shut down.
W20250904 19:58:18.371287 13801 consensus_peers.cc:489] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 -> Peer 566e42931fac46a68901dc3725fe14d7 (127.10.24.129:44369): Couldn't send request to peer 566e42931fac46a68901dc3725fe14d7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20250904 19:58:18.372391 13927 ksck_checksum.cc:286] failed to submit task to start additional tablet checksums: Service unavailable: The pool has been shut down.
I20250904 19:58:18.452914 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:18.453680 10338 tablet_replica.cc:331] T 1f59f407dece4ea1a6ebddeccc2b98ee P 566e42931fac46a68901dc3725fe14d7: stopping tablet replica
I20250904 19:58:18.454496 10338 raft_consensus.cc:2241] T 1f59f407dece4ea1a6ebddeccc2b98ee P 566e42931fac46a68901dc3725fe14d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:18.455101 10338 raft_consensus.cc:2270] T 1f59f407dece4ea1a6ebddeccc2b98ee P 566e42931fac46a68901dc3725fe14d7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250904 19:58:18.473378 13801 consensus_peers.cc:489] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 -> Peer 566e42931fac46a68901dc3725fe14d7 (127.10.24.129:44369): Couldn't send request to peer 566e42931fac46a68901dc3725fe14d7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250904 19:58:18.500968 10338 tablet_replica.cc:331] T 554fd682ad784eb7921a842bc8244e28 P 566e42931fac46a68901dc3725fe14d7: stopping tablet replica
I20250904 19:58:18.501828 10338 raft_consensus.cc:2241] T 554fd682ad784eb7921a842bc8244e28 P 566e42931fac46a68901dc3725fe14d7 [term 1 FOLLOWER]: Raft consensus shutting down.
W20250904 19:58:18.505771 13723 consensus_peers.cc:489] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 -> Peer 566e42931fac46a68901dc3725fe14d7 (127.10.24.129:44369): Couldn't send request to peer 566e42931fac46a68901dc3725fe14d7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250904 19:58:18.502444 10338 raft_consensus.cc:2270] T 554fd682ad784eb7921a842bc8244e28 P 566e42931fac46a68901dc3725fe14d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:18.512176 10338 tablet_replica.cc:331] T 4d35cb50e1a44fe19c7f168e4658586d P 566e42931fac46a68901dc3725fe14d7: stopping tablet replica
I20250904 19:58:18.512889 10338 raft_consensus.cc:2241] T 4d35cb50e1a44fe19c7f168e4658586d P 566e42931fac46a68901dc3725fe14d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:18.513481 10338 raft_consensus.cc:2270] T 4d35cb50e1a44fe19c7f168e4658586d P 566e42931fac46a68901dc3725fe14d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:18.546561 10338 tablet_server.cc:195] TabletServer@127.10.24.129:0 shutdown complete.
I20250904 19:58:18.575476 10338 tablet_server.cc:178] TabletServer@127.10.24.130:0 shutting down...
W20250904 19:58:18.652546 13927 ksck_checksum.cc:286] failed to submit task to start additional tablet checksums: Service unavailable: The pool has been shut down.
W20250904 19:58:18.698141 13926 ksck_checksum.cc:286] failed to submit task to start additional tablet checksums: Service unavailable: The pool has been shut down.
I20250904 19:58:18.759336 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:18.760531 10338 tablet_replica.cc:331] T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06: stopping tablet replica
I20250904 19:58:18.761242 10338 raft_consensus.cc:2241] T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:18.762131 10338 raft_consensus.cc:2270] T 1f59f407dece4ea1a6ebddeccc2b98ee P ab6ec379ac4a4790b8636bc0ab141e06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:18.786707 10338 tablet_replica.cc:331] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06: stopping tablet replica
I20250904 19:58:18.787489 10338 raft_consensus.cc:2241] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:18.788465 10338 raft_consensus.cc:2270] T 554fd682ad784eb7921a842bc8244e28 P ab6ec379ac4a4790b8636bc0ab141e06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:18.790691 10338 tablet_replica.cc:331] T 4d35cb50e1a44fe19c7f168e4658586d P ab6ec379ac4a4790b8636bc0ab141e06: stopping tablet replica
I20250904 19:58:18.791298 10338 raft_consensus.cc:2241] T 4d35cb50e1a44fe19c7f168e4658586d P ab6ec379ac4a4790b8636bc0ab141e06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:18.791864 10338 raft_consensus.cc:2270] T 4d35cb50e1a44fe19c7f168e4658586d P ab6ec379ac4a4790b8636bc0ab141e06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:18.808977 10338 tablet_server.cc:195] TabletServer@127.10.24.130:0 shutdown complete.
I20250904 19:58:18.826928 10338 tablet_server.cc:178] TabletServer@127.10.24.131:0 shutting down...
W20250904 19:58:18.966207 13799 consensus_peers.cc:489] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 -> Peer ab6ec379ac4a4790b8636bc0ab141e06 (127.10.24.130:34573): Couldn't send request to peer ab6ec379ac4a4790b8636bc0ab141e06. Status: Network error: Client connection negotiation failed: client connection to 127.10.24.130:34573: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250904 19:58:18.974673 13799 consensus_peers.cc:489] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 -> Peer ab6ec379ac4a4790b8636bc0ab141e06 (127.10.24.130:34573): Couldn't send request to peer ab6ec379ac4a4790b8636bc0ab141e06. Status: Network error: Client connection negotiation failed: client connection to 127.10.24.130:34573: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 19:58:19.016418 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:19.017051 10338 tablet_replica.cc:331] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942: stopping tablet replica
I20250904 19:58:19.017787 10338 raft_consensus.cc:2241] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:19.018719 10338 raft_consensus.cc:2270] T 4d35cb50e1a44fe19c7f168e4658586d P d1674ffee63e4a2ba7debbd08af98942 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:19.021152 10338 tablet_replica.cc:331] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942: stopping tablet replica
I20250904 19:58:19.021759 10338 raft_consensus.cc:2241] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:19.022269 10338 raft_consensus.cc:2270] T 554fd682ad784eb7921a842bc8244e28 P d1674ffee63e4a2ba7debbd08af98942 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:19.024273 10338 tablet_replica.cc:331] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942: stopping tablet replica
I20250904 19:58:19.024693 10338 raft_consensus.cc:2241] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:19.025350 10338 raft_consensus.cc:2270] T 1f59f407dece4ea1a6ebddeccc2b98ee P d1674ffee63e4a2ba7debbd08af98942 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250904 19:58:19.838532 13507 debug-util.cc:398] Leaking SignalData structure 0x7b08001efa20 after lost signal to thread 10341
W20250904 19:58:19.840786 13507 debug-util.cc:398] Leaking SignalData structure 0x7b08001c4b00 after lost signal to thread 13445
W20250904 19:58:19.842271 13507 debug-util.cc:398] Leaking SignalData structure 0x7b0800137f80 after lost signal to thread 13510
W20250904 19:58:19.843873 13507 debug-util.cc:398] Leaking SignalData structure 0x7b0800190e00 after lost signal to thread 13577
W20250904 19:58:20.054801 10338 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 13857)
W20250904 19:58:21.055375 10338 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 13857)
I20250904 19:58:22.041831 10338 tablet_server.cc:195] TabletServer@127.10.24.131:0 shutdown complete.
I20250904 19:58:22.069751 10338 master.cc:561] Master@127.10.24.190:40239 shutting down...
I20250904 19:58:22.099436 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:22.100867 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:22.101351 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P f6df0c299625444596e7565436e9c042: stopping tablet replica
I20250904 19:58:22.123692 10338 master.cc:583] Master@127.10.24.190:40239 shutdown complete.
I20250904 19:58:22.144093 10338 master.cc:561] Master@127.10.24.189:46123 shutting down...
I20250904 19:58:22.179889 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:22.180703 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:22.181102 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P e125eccd412340bfb2e1e817d48ebda5: stopping tablet replica
I20250904 19:58:23.043979 14029 raft_consensus.cc:491] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f6df0c299625444596e7565436e9c042)
I20250904 19:58:23.044476 14029 raft_consensus.cc:513] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6df0c299625444596e7565436e9c042" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 40239 } } peers { permanent_uuid: "e125eccd412340bfb2e1e817d48ebda5" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 46123 } } peers { permanent_uuid: "8c543ae5e9af4e52ab824d52f0097388" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 44757 } }
I20250904 19:58:23.046519 14029 leader_election.cc:290] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f6df0c299625444596e7565436e9c042 (127.10.24.190:40239), e125eccd412340bfb2e1e817d48ebda5 (127.10.24.189:46123)
W20250904 19:58:23.052130 13525 leader_election.cc:336] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f6df0c299625444596e7565436e9c042 (127.10.24.190:40239): Network error: Client connection negotiation failed: client connection to 127.10.24.190:40239: connect: Connection refused (error 111)
W20250904 19:58:23.052634 13527 leader_election.cc:336] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer e125eccd412340bfb2e1e817d48ebda5 (127.10.24.189:46123): Network error: Client connection negotiation failed: client connection to 127.10.24.189:46123: connect: Connection refused (error 111)
I20250904 19:58:23.052968 13527 leader_election.cc:304] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8c543ae5e9af4e52ab824d52f0097388; no voters: e125eccd412340bfb2e1e817d48ebda5, f6df0c299625444596e7565436e9c042
I20250904 19:58:23.053546 14029 raft_consensus.cc:2747] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250904 19:58:23.199364 10338 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 13507)
I20250904 19:58:23.241385 10338 master.cc:583] Master@127.10.24.189:46123 shutdown complete.
I20250904 19:58:23.254985 10338 master.cc:561] Master@127.10.24.188:44757 shutting down...
I20250904 19:58:23.273888 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:23.274418 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:23.274677 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 8c543ae5e9af4e52ab824d52f0097388: stopping tablet replica
I20250904 19:58:23.294942 10338 master.cc:583] Master@127.10.24.188:44757 shutdown complete.
[ OK ] RemoteKsckTest.TestChecksumTimeout (13971 ms)
[ RUN ] RemoteKsckTest.TestChecksumSnapshot
I20250904 19:58:23.341809 10338 test_util.cc:276] Using random seed: 617123032
I20250904 19:58:23.343164 10338 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.24.190:37167,127.10.24.189:37045,127.10.24.188:41447
I20250904 19:58:23.344838 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:23.351389 14033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:23.351415 14034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:23.352682 14036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:23.353967 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:23.355321 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:23.355597 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:23.355775 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015903355756 us; error 0 us; skew 500 ppm
I20250904 19:58:23.356417 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:23.359306 10338 webserver.cc:480] Webserver started at http://127.10.24.190:39811/ using document root <none> and password file <none>
I20250904 19:58:23.359916 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:23.360126 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:23.360402 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:23.361953 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/master-0-root/instance:
uuid: "0b3ae42117f64a1798aa4441b428c33e"
format_stamp: "Formatted at 2025-09-04 19:58:23 on dist-test-slave-f3xj"
I20250904 19:58:23.366976 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20250904 19:58:23.370023 14041 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:23.370657 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250904 19:58:23.370906 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/master-0-root
uuid: "0b3ae42117f64a1798aa4441b428c33e"
format_stamp: "Formatted at 2025-09-04 19:58:23 on dist-test-slave-f3xj"
I20250904 19:58:23.371140 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:23.391613 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:23.392706 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:23.428530 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.190:37167
I20250904 19:58:23.428618 14092 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.190:37167 every 8 connection(s)
I20250904 19:58:23.432340 14093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:23.432462 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:23.437626 14095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:23.439899 14096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:23.440474 14098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:23.441401 14093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:23.442658 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:23.444360 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:23.444607 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:23.444833 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015903444815 us; error 0 us; skew 500 ppm
I20250904 19:58:23.445467 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:23.447940 10338 webserver.cc:480] Webserver started at http://127.10.24.189:33707/ using document root <none> and password file <none>
I20250904 19:58:23.448378 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:23.448540 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:23.448760 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:23.449805 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/master-1-root/instance:
uuid: "84f62af8f4a042049577f0ae62b44d13"
format_stamp: "Formatted at 2025-09-04 19:58:23 on dist-test-slave-f3xj"
I20250904 19:58:23.453169 14093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:23.454381 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
W20250904 19:58:23.455158 14045 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.24.189:37045: connect: Connection refused (error 111) [suppressed 8 similar messages]
W20250904 19:58:23.458156 14093 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.189:37045: Network error: Client connection negotiation failed: client connection to 127.10.24.189:37045: connect: Connection refused (error 111)
I20250904 19:58:23.458715 14106 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:23.459384 10338 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20250904 19:58:23.459650 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/master-1-root
uuid: "84f62af8f4a042049577f0ae62b44d13"
format_stamp: "Formatted at 2025-09-04 19:58:23 on dist-test-slave-f3xj"
I20250904 19:58:23.459888 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:23.473905 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:23.474921 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:23.482635 14093 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } attempt: 1
W20250904 19:58:23.486855 14093 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.189:37045: Network error: Client connection negotiation failed: client connection to 127.10.24.189:37045: connect: Connection refused (error 111)
I20250904 19:58:23.512001 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.189:37045
I20250904 19:58:23.512100 14157 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.189:37045 every 8 connection(s)
I20250904 19:58:23.515789 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:23.515802 14158 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20250904 19:58:23.521113 14160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:23.521634 14161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:23.523968 14163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:23.525311 14158 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:23.529858 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:23.530687 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:23.530874 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:23.531028 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015903531012 us; error 0 us; skew 500 ppm
I20250904 19:58:23.531502 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:23.533334 14093 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } attempt: 2
I20250904 19:58:23.534839 10338 webserver.cc:480] Webserver started at http://127.10.24.188:37899/ using document root <none> and password file <none>
I20250904 19:58:23.535483 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:23.535728 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:23.536021 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:23.537317 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/master-2-root/instance:
uuid: "709db88c60f648b0b916f3c7e202305f"
format_stamp: "Formatted at 2025-09-04 19:58:23 on dist-test-slave-f3xj"
I20250904 19:58:23.538452 14158 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:23.543471 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20250904 19:58:23.548017 14093 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:23.548516 14171 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:23.549311 10338 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20250904 19:58:23.549615 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/master-2-root
uuid: "709db88c60f648b0b916f3c7e202305f"
format_stamp: "Formatted at 2025-09-04 19:58:23 on dist-test-slave-f3xj"
I20250904 19:58:23.549932 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:23.550796 14158 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:58:23.553052 14093 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:41447: Network error: Client connection negotiation failed: client connection to 127.10.24.188:41447: connect: Connection refused (error 111)
W20250904 19:58:23.556624 14158 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:41447: Network error: Client connection negotiation failed: client connection to 127.10.24.188:41447: connect: Connection refused (error 111)
I20250904 19:58:23.576514 14093 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } attempt: 1
W20250904 19:58:23.579919 14093 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:41447: Network error: Client connection negotiation failed: client connection to 127.10.24.188:41447: connect: Connection refused (error 111)
I20250904 19:58:23.583052 14158 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } attempt: 1
I20250904 19:58:23.583405 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:23.584746 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
W20250904 19:58:23.586822 14158 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:41447: Network error: Client connection negotiation failed: client connection to 127.10.24.188:41447: connect: Connection refused (error 111)
I20250904 19:58:23.621393 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.188:41447
I20250904 19:58:23.621482 14223 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.188:41447 every 8 connection(s)
I20250904 19:58:23.624377 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 19:58:23.625084 14224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:23.631472 14224 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:23.637291 14093 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } attempt: 2
I20250904 19:58:23.641934 14224 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:23.651700 14224 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:23.653709 14093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e: Bootstrap starting.
I20250904 19:58:23.658454 14093 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:23.662911 14093 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e: No bootstrap required, opened a new log
I20250904 19:58:23.665213 14224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f: Bootstrap starting.
I20250904 19:58:23.665206 14093 raft_consensus.cc:357] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } } peers { permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } } peers { permanent_uuid: "709db88c60f648b0b916f3c7e202305f" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } }
I20250904 19:58:23.665940 14093 raft_consensus.cc:383] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:23.666160 14093 raft_consensus.cc:738] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b3ae42117f64a1798aa4441b428c33e, State: Initialized, Role: FOLLOWER
I20250904 19:58:23.666687 14093 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } } peers { permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } } peers { permanent_uuid: "709db88c60f648b0b916f3c7e202305f" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } }
I20250904 19:58:23.668300 14158 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } attempt: 2
I20250904 19:58:23.668576 14229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } } peers { permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } } peers { permanent_uuid: "709db88c60f648b0b916f3c7e202305f" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } } }
I20250904 19:58:23.669322 14229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:23.670131 14093 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:23.671008 14224 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:23.676523 14224 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f: No bootstrap required, opened a new log
I20250904 19:58:23.678979 14224 raft_consensus.cc:357] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } } peers { permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } } peers { permanent_uuid: "709db88c60f648b0b916f3c7e202305f" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } }
I20250904 19:58:23.679662 14224 raft_consensus.cc:383] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:23.679919 14224 raft_consensus.cc:738] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 709db88c60f648b0b916f3c7e202305f, State: Initialized, Role: FOLLOWER
I20250904 19:58:23.680622 14224 consensus_queue.cc:260] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } } peers { permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } } peers { permanent_uuid: "709db88c60f648b0b916f3c7e202305f" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } }
I20250904 19:58:23.682749 14237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } } peers { permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } } peers { permanent_uuid: "709db88c60f648b0b916f3c7e202305f" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } } }
I20250904 19:58:23.683589 14237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:23.684224 14224 sys_catalog.cc:564] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:23.688477 14158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13: Bootstrap starting.
I20250904 19:58:23.688828 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
W20250904 19:58:23.692251 14243 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:58:23.692513 14243 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250904 19:58:23.695547 14158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:23.700510 14158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13: No bootstrap required, opened a new log
I20250904 19:58:23.702450 14158 raft_consensus.cc:357] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } } peers { permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } } peers { permanent_uuid: "709db88c60f648b0b916f3c7e202305f" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } }
I20250904 19:58:23.702924 14158 raft_consensus.cc:383] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:23.703135 14158 raft_consensus.cc:738] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84f62af8f4a042049577f0ae62b44d13, State: Initialized, Role: FOLLOWER
I20250904 19:58:23.703680 14158 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } } peers { permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } } peers { permanent_uuid: "709db88c60f648b0b916f3c7e202305f" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } }
W20250904 19:58:23.704137 14254 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:58:23.704406 14254 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250904 19:58:23.705235 14255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } } peers { permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } } peers { permanent_uuid: "709db88c60f648b0b916f3c7e202305f" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } } }
I20250904 19:58:23.705717 14255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:23.706331 14158 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:23.715739 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20250904 19:58:23.717475 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:23.719619 14266 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:58:23.719945 14266 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20250904 19:58:23.724313 14267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:23.724866 14268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:23.726097 14270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:23.726487 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:23.727192 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:23.727384 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:23.727535 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015903727520 us; error 0 us; skew 500 ppm
W20250904 19:58:23.727658 14112 tablet.cc:2378] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250904 19:58:23.728312 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:23.730410 10338 webserver.cc:480] Webserver started at http://127.10.24.129:35413/ using document root <none> and password file <none>
I20250904 19:58:23.730837 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:23.730993 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:23.731187 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:23.732256 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/ts-0-root/instance:
uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532"
format_stamp: "Formatted at 2025-09-04 19:58:23 on dist-test-slave-f3xj"
I20250904 19:58:23.736707 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20250904 19:58:23.740330 14275 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:23.740990 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20250904 19:58:23.741230 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/ts-0-root
uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532"
format_stamp: "Formatted at 2025-09-04 19:58:23 on dist-test-slave-f3xj"
I20250904 19:58:23.741447 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:23.751137 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:23.752328 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:23.753576 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:23.755659 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:23.755828 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:23.756059 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:23.756212 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:23.793882 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.129:36517
I20250904 19:58:23.793980 14337 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.129:36517 every 8 connection(s)
I20250904 19:58:23.808488 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:23.817756 14338 heartbeater.cc:344] Connected to a master server at 127.10.24.189:37045
I20250904 19:58:23.818303 14338 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:23.819295 14338 heartbeater.cc:507] Master 127.10.24.189:37045 requested a full tablet report, sending...
W20250904 19:58:23.820705 14346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:23.821499 14339 heartbeater.cc:344] Connected to a master server at 127.10.24.188:41447
I20250904 19:58:23.821921 14339 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:23.822319 14123 ts_manager.cc:194] Registered new tserver with Master: 9e5b52e72bcc448dbd6cc93d8fcf8532 (127.10.24.129:36517)
I20250904 19:58:23.822842 14339 heartbeater.cc:507] Master 127.10.24.188:41447 requested a full tablet report, sending...
W20250904 19:58:23.824525 14345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:23.825829 14189 ts_manager.cc:194] Registered new tserver with Master: 9e5b52e72bcc448dbd6cc93d8fcf8532 (127.10.24.129:36517)
W20250904 19:58:23.827471 14348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:23.829268 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:23.829587 14340 heartbeater.cc:344] Connected to a master server at 127.10.24.190:37167
I20250904 19:58:23.829942 14340 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:23.830156 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:23.830384 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:23.830536 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015903830522 us; error 0 us; skew 500 ppm
I20250904 19:58:23.830585 14340 heartbeater.cc:507] Master 127.10.24.190:37167 requested a full tablet report, sending...
I20250904 19:58:23.831102 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:23.833253 14058 ts_manager.cc:194] Registered new tserver with Master: 9e5b52e72bcc448dbd6cc93d8fcf8532 (127.10.24.129:36517)
I20250904 19:58:23.833719 10338 webserver.cc:480] Webserver started at http://127.10.24.130:39905/ using document root <none> and password file <none>
I20250904 19:58:23.834358 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:23.834600 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:23.834908 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:23.836447 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/ts-1-root/instance:
uuid: "2530a39884164942bc417e0905a098f9"
format_stamp: "Formatted at 2025-09-04 19:58:23 on dist-test-slave-f3xj"
I20250904 19:58:23.840907 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250904 19:58:23.844043 14353 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:23.844789 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250904 19:58:23.845132 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/ts-1-root
uuid: "2530a39884164942bc417e0905a098f9"
format_stamp: "Formatted at 2025-09-04 19:58:23 on dist-test-slave-f3xj"
I20250904 19:58:23.845422 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:23.866627 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:23.867766 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:23.869073 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:23.871261 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:23.871457 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:23.871708 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:23.871910 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:23.878111 14237 raft_consensus.cc:491] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:23.878589 14237 raft_consensus.cc:513] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } } peers { permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } } peers { permanent_uuid: "709db88c60f648b0b916f3c7e202305f" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } }
I20250904 19:58:23.881006 14237 leader_election.cc:290] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0b3ae42117f64a1798aa4441b428c33e (127.10.24.190:37167), 84f62af8f4a042049577f0ae62b44d13 (127.10.24.189:37045)
I20250904 19:58:23.882002 14068 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "709db88c60f648b0b916f3c7e202305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b3ae42117f64a1798aa4441b428c33e" is_pre_election: true
I20250904 19:58:23.882761 14133 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "709db88c60f648b0b916f3c7e202305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84f62af8f4a042049577f0ae62b44d13" is_pre_election: true
I20250904 19:58:23.882768 14068 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 709db88c60f648b0b916f3c7e202305f in term 0.
I20250904 19:58:23.884503 14133 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 709db88c60f648b0b916f3c7e202305f in term 0.
I20250904 19:58:23.884474 14172 leader_election.cc:304] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b3ae42117f64a1798aa4441b428c33e, 709db88c60f648b0b916f3c7e202305f; no voters:
I20250904 19:58:23.886415 14237 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:23.886791 14237 raft_consensus.cc:491] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:23.887118 14237 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:23.895362 14237 raft_consensus.cc:513] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } } peers { permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } } peers { permanent_uuid: "709db88c60f648b0b916f3c7e202305f" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } }
I20250904 19:58:23.898932 14068 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "709db88c60f648b0b916f3c7e202305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0b3ae42117f64a1798aa4441b428c33e"
I20250904 19:58:23.899667 14068 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:23.900648 14237 leader_election.cc:290] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [CANDIDATE]: Term 1 election: Requested vote from peers 0b3ae42117f64a1798aa4441b428c33e (127.10.24.190:37167), 84f62af8f4a042049577f0ae62b44d13 (127.10.24.189:37045)
I20250904 19:58:23.902756 14133 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "709db88c60f648b0b916f3c7e202305f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84f62af8f4a042049577f0ae62b44d13"
I20250904 19:58:23.903378 14133 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:23.906839 14068 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 709db88c60f648b0b916f3c7e202305f in term 1.
I20250904 19:58:23.909376 14172 leader_election.cc:304] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b3ae42117f64a1798aa4441b428c33e, 709db88c60f648b0b916f3c7e202305f; no voters:
I20250904 19:58:23.910291 14237 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:23.911916 14133 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 709db88c60f648b0b916f3c7e202305f in term 1.
I20250904 19:58:23.913508 14237 raft_consensus.cc:695] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [term 1 LEADER]: Becoming Leader. State: Replica: 709db88c60f648b0b916f3c7e202305f, State: Running, Role: LEADER
I20250904 19:58:23.914475 14237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } } peers { permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } } peers { permanent_uuid: "709db88c60f648b0b916f3c7e202305f" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } }
I20250904 19:58:23.921006 14386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 709db88c60f648b0b916f3c7e202305f. Latest consensus state: current_term: 1 leader_uuid: "709db88c60f648b0b916f3c7e202305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } } peers { permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } } peers { permanent_uuid: "709db88c60f648b0b916f3c7e202305f" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } } }
I20250904 19:58:23.921799 14386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:23.924047 14398 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 19:58:23.930766 14398 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 19:58:23.940878 14133 raft_consensus.cc:1273] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13 [term 1 FOLLOWER]: Refusing update from remote peer 709db88c60f648b0b916f3c7e202305f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:23.941311 14068 raft_consensus.cc:1273] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e [term 1 FOLLOWER]: Refusing update from remote peer 709db88c60f648b0b916f3c7e202305f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:23.943499 14237 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:23.945366 14237 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:23.946681 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.130:39079
I20250904 19:58:23.947295 14419 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.130:39079 every 8 connection(s)
I20250904 19:58:23.954669 14229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 709db88c60f648b0b916f3c7e202305f. Latest consensus state: current_term: 1 leader_uuid: "709db88c60f648b0b916f3c7e202305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } } peers { permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } } peers { permanent_uuid: "709db88c60f648b0b916f3c7e202305f" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } } }
I20250904 19:58:23.955423 14229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:23.957626 14255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 709db88c60f648b0b916f3c7e202305f. Latest consensus state: current_term: 1 leader_uuid: "709db88c60f648b0b916f3c7e202305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } } peers { permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } } peers { permanent_uuid: "709db88c60f648b0b916f3c7e202305f" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } } }
I20250904 19:58:23.958333 14255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:23.974452 14237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "709db88c60f648b0b916f3c7e202305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } } peers { permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } } peers { permanent_uuid: "709db88c60f648b0b916f3c7e202305f" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } } }
I20250904 19:58:23.975298 14237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:23.976145 14237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "709db88c60f648b0b916f3c7e202305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } } peers { permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } } peers { permanent_uuid: "709db88c60f648b0b916f3c7e202305f" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } } }
I20250904 19:58:23.976823 14237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:23.987236 14398 catalog_manager.cc:1349] Generated new cluster ID: 7c14ce8cffdc47a0aebecf0bb1f67fe4
I20250904 19:58:23.987607 14398 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 19:58:23.994637 14229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "709db88c60f648b0b916f3c7e202305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } } peers { permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } } peers { permanent_uuid: "709db88c60f648b0b916f3c7e202305f" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } } }
I20250904 19:58:23.995395 14229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:24.024673 14422 heartbeater.cc:344] Connected to a master server at 127.10.24.189:37045
I20250904 19:58:24.025170 14422 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:24.026212 14422 heartbeater.cc:507] Master 127.10.24.189:37045 requested a full tablet report, sending...
I20250904 19:58:24.057682 14427 heartbeater.cc:344] Connected to a master server at 127.10.24.188:41447
I20250904 19:58:24.058101 14427 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:24.058918 14427 heartbeater.cc:507] Master 127.10.24.188:41447 requested a full tablet report, sending...
I20250904 19:58:24.060999 14189 ts_manager.cc:194] Registered new tserver with Master: 2530a39884164942bc417e0905a098f9 (127.10.24.130:39079)
I20250904 19:58:24.060734 14255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "709db88c60f648b0b916f3c7e202305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } } peers { permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } } peers { permanent_uuid: "709db88c60f648b0b916f3c7e202305f" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } } }
I20250904 19:58:24.061558 14255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:24.065773 14122 ts_manager.cc:194] Registered new tserver with Master: 2530a39884164942bc417e0905a098f9 (127.10.24.130:39079)
I20250904 19:58:24.072172 14429 heartbeater.cc:344] Connected to a master server at 127.10.24.190:37167
I20250904 19:58:24.072549 14429 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:24.073377 14429 heartbeater.cc:507] Master 127.10.24.190:37167 requested a full tablet report, sending...
I20250904 19:58:24.075659 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:24.079270 14058 ts_manager.cc:194] Registered new tserver with Master: 2530a39884164942bc417e0905a098f9 (127.10.24.130:39079)
I20250904 19:58:24.087770 14398 catalog_manager.cc:1372] Generated new certificate authority record
W20250904 19:58:24.089304 14434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:24.092852 14398 catalog_manager.cc:1506] Loading token signing keys...
W20250904 19:58:24.094549 14435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:24.106562 14438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:24.106688 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:24.107671 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:24.107892 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:24.108068 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015904108055 us; error 0 us; skew 500 ppm
I20250904 19:58:24.108664 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:24.119513 10338 webserver.cc:480] Webserver started at http://127.10.24.131:41883/ using document root <none> and password file <none>
I20250904 19:58:24.120183 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:24.120419 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:24.120729 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:24.121815 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/ts-2-root/instance:
uuid: "1b540f5979d7433cba2899c26c907c6c"
format_stamp: "Formatted at 2025-09-04 19:58:24 on dist-test-slave-f3xj"
I20250904 19:58:24.127686 14398 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f: Generated new TSK 0
I20250904 19:58:24.128376 14398 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 19:58:24.138053 14189 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54610
I20250904 19:58:24.139477 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.016s user 0.000s sys 0.010s
I20250904 19:58:24.139967 14188 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54616
I20250904 19:58:24.142838 14443 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:24.143626 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250904 19:58:24.143961 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/ts-2-root
uuid: "1b540f5979d7433cba2899c26c907c6c"
format_stamp: "Formatted at 2025-09-04 19:58:24 on dist-test-slave-f3xj"
I20250904 19:58:24.144263 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshot.1757015869512910-10338-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:24.150661 14339 heartbeater.cc:499] Master 127.10.24.188:41447 was elected leader, sending a full tablet report...
I20250904 19:58:24.152081 14427 heartbeater.cc:499] Master 127.10.24.188:41447 was elected leader, sending a full tablet report...
I20250904 19:58:24.160957 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:24.162411 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:24.163867 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:24.166976 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:24.167179 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:24.167448 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:24.167663 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20250904 19:58:24.232537 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.131:33049
I20250904 19:58:24.232647 14505 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.131:33049 every 8 connection(s)
I20250904 19:58:24.266626 14506 heartbeater.cc:344] Connected to a master server at 127.10.24.189:37045
I20250904 19:58:24.267071 14506 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:24.267941 14506 heartbeater.cc:507] Master 127.10.24.189:37045 requested a full tablet report, sending...
I20250904 19:58:24.270453 14123 ts_manager.cc:194] Registered new tserver with Master: 1b540f5979d7433cba2899c26c907c6c (127.10.24.131:33049)
I20250904 19:58:24.283324 14508 heartbeater.cc:344] Connected to a master server at 127.10.24.190:37167
I20250904 19:58:24.283785 14508 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:24.284646 14508 heartbeater.cc:507] Master 127.10.24.190:37167 requested a full tablet report, sending...
I20250904 19:58:24.285732 14507 heartbeater.cc:344] Connected to a master server at 127.10.24.188:41447
I20250904 19:58:24.286056 14507 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:24.286837 14507 heartbeater.cc:507] Master 127.10.24.188:41447 requested a full tablet report, sending...
I20250904 19:58:24.287312 14056 ts_manager.cc:194] Registered new tserver with Master: 1b540f5979d7433cba2899c26c907c6c (127.10.24.131:33049)
I20250904 19:58:24.288766 14188 ts_manager.cc:194] Registered new tserver with Master: 1b540f5979d7433cba2899c26c907c6c (127.10.24.131:33049)
I20250904 19:58:24.289059 10338 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019059523s
I20250904 19:58:24.290431 14188 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54620
I20250904 19:58:24.303619 14507 heartbeater.cc:499] Master 127.10.24.188:41447 was elected leader, sending a full tablet report...
I20250904 19:58:24.344416 14189 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54626:
name: "ksck-test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\n\000\000\000\004\001\024\000\000\000""\004\001\n\000\000\000\004\001\024\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 19:58:24.347085 14189 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table ksck-test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 19:58:24.443971 14469 tablet_service.cc:1468] Processing CreateTablet for tablet 3c4ea89dcf084fba8a4f5f8e73cdfdc4 (DEFAULT_TABLE table=ksck-test-table [id=f58835533b3b4a4a89286ca47a25bbee]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:24.445398 14469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c4ea89dcf084fba8a4f5f8e73cdfdc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:24.446370 14303 tablet_service.cc:1468] Processing CreateTablet for tablet 334c941721634f9c85c1e0b6a09f4611 (DEFAULT_TABLE table=ksck-test-table [id=f58835533b3b4a4a89286ca47a25bbee]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:24.447516 14303 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 334c941721634f9c85c1e0b6a09f4611. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:24.449905 14302 tablet_service.cc:1468] Processing CreateTablet for tablet dea61aee32184c449f468ef09dbfd1ec (DEFAULT_TABLE table=ksck-test-table [id=f58835533b3b4a4a89286ca47a25bbee]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:24.450951 14302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dea61aee32184c449f468ef09dbfd1ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:24.452514 14301 tablet_service.cc:1468] Processing CreateTablet for tablet 3c4ea89dcf084fba8a4f5f8e73cdfdc4 (DEFAULT_TABLE table=ksck-test-table [id=f58835533b3b4a4a89286ca47a25bbee]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:24.453670 14301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c4ea89dcf084fba8a4f5f8e73cdfdc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:24.457193 14381 tablet_service.cc:1468] Processing CreateTablet for tablet 334c941721634f9c85c1e0b6a09f4611 (DEFAULT_TABLE table=ksck-test-table [id=f58835533b3b4a4a89286ca47a25bbee]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:24.458319 14381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 334c941721634f9c85c1e0b6a09f4611. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:24.459900 14471 tablet_service.cc:1468] Processing CreateTablet for tablet 334c941721634f9c85c1e0b6a09f4611 (DEFAULT_TABLE table=ksck-test-table [id=f58835533b3b4a4a89286ca47a25bbee]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:24.460783 14471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 334c941721634f9c85c1e0b6a09f4611. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:24.444561 14470 tablet_service.cc:1468] Processing CreateTablet for tablet dea61aee32184c449f468ef09dbfd1ec (DEFAULT_TABLE table=ksck-test-table [id=f58835533b3b4a4a89286ca47a25bbee]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:24.466773 14470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dea61aee32184c449f468ef09dbfd1ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:24.472429 14380 tablet_service.cc:1468] Processing CreateTablet for tablet dea61aee32184c449f468ef09dbfd1ec (DEFAULT_TABLE table=ksck-test-table [id=f58835533b3b4a4a89286ca47a25bbee]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:24.473521 14380 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dea61aee32184c449f468ef09dbfd1ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:24.477264 14527 tablet_bootstrap.cc:492] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c: Bootstrap starting.
I20250904 19:58:24.489055 14527 tablet_bootstrap.cc:654] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:24.491353 14379 tablet_service.cc:1468] Processing CreateTablet for tablet 3c4ea89dcf084fba8a4f5f8e73cdfdc4 (DEFAULT_TABLE table=ksck-test-table [id=f58835533b3b4a4a89286ca47a25bbee]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:24.492669 14379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c4ea89dcf084fba8a4f5f8e73cdfdc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:24.513024 14527 tablet_bootstrap.cc:492] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c: No bootstrap required, opened a new log
I20250904 19:58:24.513528 14527 ts_tablet_manager.cc:1397] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c: Time spent bootstrapping tablet: real 0.037s user 0.005s sys 0.007s
I20250904 19:58:24.519037 14527 raft_consensus.cc:357] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } }
I20250904 19:58:24.519896 14527 raft_consensus.cc:383] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:24.520269 14527 raft_consensus.cc:738] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b540f5979d7433cba2899c26c907c6c, State: Initialized, Role: FOLLOWER
I20250904 19:58:24.521083 14527 consensus_queue.cc:260] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } }
I20250904 19:58:24.525683 14529 tablet_bootstrap.cc:492] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9: Bootstrap starting.
I20250904 19:58:24.529579 14530 tablet_bootstrap.cc:492] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532: Bootstrap starting.
I20250904 19:58:24.534806 14527 ts_tablet_manager.cc:1428] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c: Time spent starting tablet: real 0.021s user 0.009s sys 0.002s
I20250904 19:58:24.543321 14527 tablet_bootstrap.cc:492] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 1b540f5979d7433cba2899c26c907c6c: Bootstrap starting.
I20250904 19:58:24.549281 14527 tablet_bootstrap.cc:654] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 1b540f5979d7433cba2899c26c907c6c: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:24.560350 14527 tablet_bootstrap.cc:492] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 1b540f5979d7433cba2899c26c907c6c: No bootstrap required, opened a new log
I20250904 19:58:24.560910 14527 ts_tablet_manager.cc:1397] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 1b540f5979d7433cba2899c26c907c6c: Time spent bootstrapping tablet: real 0.018s user 0.016s sys 0.000s
I20250904 19:58:24.564288 14527 raft_consensus.cc:357] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 1b540f5979d7433cba2899c26c907c6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } }
I20250904 19:58:24.567772 14530 tablet_bootstrap.cc:654] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:24.568053 14527 raft_consensus.cc:383] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 1b540f5979d7433cba2899c26c907c6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:24.568336 14527 raft_consensus.cc:738] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 1b540f5979d7433cba2899c26c907c6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b540f5979d7433cba2899c26c907c6c, State: Initialized, Role: FOLLOWER
I20250904 19:58:24.568992 14527 consensus_queue.cc:260] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 1b540f5979d7433cba2899c26c907c6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } }
I20250904 19:58:24.571291 14527 ts_tablet_manager.cc:1428] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 1b540f5979d7433cba2899c26c907c6c: Time spent starting tablet: real 0.010s user 0.005s sys 0.000s
I20250904 19:58:24.567824 14529 tablet_bootstrap.cc:654] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:24.576009 14527 tablet_bootstrap.cc:492] T dea61aee32184c449f468ef09dbfd1ec P 1b540f5979d7433cba2899c26c907c6c: Bootstrap starting.
I20250904 19:58:24.582335 14527 tablet_bootstrap.cc:654] T dea61aee32184c449f468ef09dbfd1ec P 1b540f5979d7433cba2899c26c907c6c: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:24.588371 14529 tablet_bootstrap.cc:492] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9: No bootstrap required, opened a new log
I20250904 19:58:24.588932 14529 ts_tablet_manager.cc:1397] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9: Time spent bootstrapping tablet: real 0.063s user 0.015s sys 0.002s
I20250904 19:58:24.591874 14529 raft_consensus.cc:357] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } }
I20250904 19:58:24.592767 14529 raft_consensus.cc:383] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:24.593118 14529 raft_consensus.cc:738] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2530a39884164942bc417e0905a098f9, State: Initialized, Role: FOLLOWER
I20250904 19:58:24.598074 14529 consensus_queue.cc:260] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } }
I20250904 19:58:24.604022 14527 tablet_bootstrap.cc:492] T dea61aee32184c449f468ef09dbfd1ec P 1b540f5979d7433cba2899c26c907c6c: No bootstrap required, opened a new log
I20250904 19:58:24.604547 14527 ts_tablet_manager.cc:1397] T dea61aee32184c449f468ef09dbfd1ec P 1b540f5979d7433cba2899c26c907c6c: Time spent bootstrapping tablet: real 0.029s user 0.017s sys 0.009s
I20250904 19:58:24.607302 14527 raft_consensus.cc:357] T dea61aee32184c449f468ef09dbfd1ec P 1b540f5979d7433cba2899c26c907c6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } }
I20250904 19:58:24.608045 14527 raft_consensus.cc:383] T dea61aee32184c449f468ef09dbfd1ec P 1b540f5979d7433cba2899c26c907c6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:24.608366 14527 raft_consensus.cc:738] T dea61aee32184c449f468ef09dbfd1ec P 1b540f5979d7433cba2899c26c907c6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b540f5979d7433cba2899c26c907c6c, State: Initialized, Role: FOLLOWER
I20250904 19:58:24.609169 14527 consensus_queue.cc:260] T dea61aee32184c449f468ef09dbfd1ec P 1b540f5979d7433cba2899c26c907c6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } }
I20250904 19:58:24.611383 14527 ts_tablet_manager.cc:1428] T dea61aee32184c449f468ef09dbfd1ec P 1b540f5979d7433cba2899c26c907c6c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250904 19:58:24.612109 14529 ts_tablet_manager.cc:1428] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9: Time spent starting tablet: real 0.023s user 0.005s sys 0.005s
I20250904 19:58:24.614895 14529 tablet_bootstrap.cc:492] T dea61aee32184c449f468ef09dbfd1ec P 2530a39884164942bc417e0905a098f9: Bootstrap starting.
I20250904 19:58:24.625943 14529 tablet_bootstrap.cc:654] T dea61aee32184c449f468ef09dbfd1ec P 2530a39884164942bc417e0905a098f9: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:24.629765 14530 tablet_bootstrap.cc:492] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532: No bootstrap required, opened a new log
I20250904 19:58:24.630182 14530 ts_tablet_manager.cc:1397] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532: Time spent bootstrapping tablet: real 0.101s user 0.008s sys 0.005s
I20250904 19:58:24.630897 14529 tablet_bootstrap.cc:492] T dea61aee32184c449f468ef09dbfd1ec P 2530a39884164942bc417e0905a098f9: No bootstrap required, opened a new log
I20250904 19:58:24.631278 14529 ts_tablet_manager.cc:1397] T dea61aee32184c449f468ef09dbfd1ec P 2530a39884164942bc417e0905a098f9: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.000s
I20250904 19:58:24.632205 14530 raft_consensus.cc:357] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } }
I20250904 19:58:24.632709 14530 raft_consensus.cc:383] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:24.632927 14530 raft_consensus.cc:738] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e5b52e72bcc448dbd6cc93d8fcf8532, State: Initialized, Role: FOLLOWER
I20250904 19:58:24.633236 14529 raft_consensus.cc:357] T dea61aee32184c449f468ef09dbfd1ec P 2530a39884164942bc417e0905a098f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } }
I20250904 19:58:24.633801 14529 raft_consensus.cc:383] T dea61aee32184c449f468ef09dbfd1ec P 2530a39884164942bc417e0905a098f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:24.633526 14530 consensus_queue.cc:260] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } }
I20250904 19:58:24.634042 14529 raft_consensus.cc:738] T dea61aee32184c449f468ef09dbfd1ec P 2530a39884164942bc417e0905a098f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2530a39884164942bc417e0905a098f9, State: Initialized, Role: FOLLOWER
I20250904 19:58:24.634840 14529 consensus_queue.cc:260] T dea61aee32184c449f468ef09dbfd1ec P 2530a39884164942bc417e0905a098f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } }
I20250904 19:58:24.641682 14529 ts_tablet_manager.cc:1428] T dea61aee32184c449f468ef09dbfd1ec P 2530a39884164942bc417e0905a098f9: Time spent starting tablet: real 0.010s user 0.003s sys 0.002s
I20250904 19:58:24.651389 14529 tablet_bootstrap.cc:492] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 2530a39884164942bc417e0905a098f9: Bootstrap starting.
I20250904 19:58:24.656941 14529 tablet_bootstrap.cc:654] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 2530a39884164942bc417e0905a098f9: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:24.658241 14530 ts_tablet_manager.cc:1428] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532: Time spent starting tablet: real 0.028s user 0.015s sys 0.000s
I20250904 19:58:24.662873 14529 tablet_bootstrap.cc:492] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 2530a39884164942bc417e0905a098f9: No bootstrap required, opened a new log
I20250904 19:58:24.663404 14529 ts_tablet_manager.cc:1397] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 2530a39884164942bc417e0905a098f9: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20250904 19:58:24.663584 14530 tablet_bootstrap.cc:492] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532: Bootstrap starting.
I20250904 19:58:24.666301 14529 raft_consensus.cc:357] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 2530a39884164942bc417e0905a098f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } }
I20250904 19:58:24.667079 14529 raft_consensus.cc:383] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 2530a39884164942bc417e0905a098f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:24.667446 14529 raft_consensus.cc:738] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 2530a39884164942bc417e0905a098f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2530a39884164942bc417e0905a098f9, State: Initialized, Role: FOLLOWER
I20250904 19:58:24.668241 14529 consensus_queue.cc:260] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 2530a39884164942bc417e0905a098f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } }
I20250904 19:58:24.670815 14529 ts_tablet_manager.cc:1428] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 2530a39884164942bc417e0905a098f9: Time spent starting tablet: real 0.007s user 0.002s sys 0.003s
I20250904 19:58:24.671231 14530 tablet_bootstrap.cc:654] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:24.682435 14530 tablet_bootstrap.cc:492] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532: No bootstrap required, opened a new log
I20250904 19:58:24.683005 14530 ts_tablet_manager.cc:1397] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.004s
I20250904 19:58:24.690117 14530 raft_consensus.cc:357] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } }
I20250904 19:58:24.690711 14530 raft_consensus.cc:383] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:24.690925 14530 raft_consensus.cc:738] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e5b52e72bcc448dbd6cc93d8fcf8532, State: Initialized, Role: FOLLOWER
I20250904 19:58:24.691457 14530 consensus_queue.cc:260] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } }
I20250904 19:58:24.693573 14530 ts_tablet_manager.cc:1428] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532: Time spent starting tablet: real 0.010s user 0.001s sys 0.003s
I20250904 19:58:24.694555 14530 tablet_bootstrap.cc:492] T 334c941721634f9c85c1e0b6a09f4611 P 9e5b52e72bcc448dbd6cc93d8fcf8532: Bootstrap starting.
I20250904 19:58:24.697360 14243 catalog_manager.cc:1261] Loaded cluster ID: 7c14ce8cffdc47a0aebecf0bb1f67fe4
I20250904 19:58:24.699975 14243 catalog_manager.cc:1554] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e: loading cluster ID for follower catalog manager: success
I20250904 19:58:24.701795 14530 tablet_bootstrap.cc:654] T 334c941721634f9c85c1e0b6a09f4611 P 9e5b52e72bcc448dbd6cc93d8fcf8532: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:24.710582 14530 tablet_bootstrap.cc:492] T 334c941721634f9c85c1e0b6a09f4611 P 9e5b52e72bcc448dbd6cc93d8fcf8532: No bootstrap required, opened a new log
I20250904 19:58:24.711045 14530 ts_tablet_manager.cc:1397] T 334c941721634f9c85c1e0b6a09f4611 P 9e5b52e72bcc448dbd6cc93d8fcf8532: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.007s
I20250904 19:58:24.713658 14530 raft_consensus.cc:357] T 334c941721634f9c85c1e0b6a09f4611 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } }
I20250904 19:58:24.714293 14530 raft_consensus.cc:383] T 334c941721634f9c85c1e0b6a09f4611 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:24.714388 14243 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e: acquiring CA information for follower catalog manager: success
I20250904 19:58:24.714605 14530 raft_consensus.cc:738] T 334c941721634f9c85c1e0b6a09f4611 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e5b52e72bcc448dbd6cc93d8fcf8532, State: Initialized, Role: FOLLOWER
I20250904 19:58:24.715320 14530 consensus_queue.cc:260] T 334c941721634f9c85c1e0b6a09f4611 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } }
I20250904 19:58:24.717459 14530 ts_tablet_manager.cc:1428] T 334c941721634f9c85c1e0b6a09f4611 P 9e5b52e72bcc448dbd6cc93d8fcf8532: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20250904 19:58:24.719075 14243 catalog_manager.cc:1604] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:58:24.725332 14266 catalog_manager.cc:1261] Loaded cluster ID: 7c14ce8cffdc47a0aebecf0bb1f67fe4
I20250904 19:58:24.725677 14266 catalog_manager.cc:1554] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13: loading cluster ID for follower catalog manager: success
I20250904 19:58:24.731796 14266 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13: acquiring CA information for follower catalog manager: success
I20250904 19:58:24.735751 14266 catalog_manager.cc:1604] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:58:24.736704 14538 raft_consensus.cc:491] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:24.737193 14538 raft_consensus.cc:513] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } }
I20250904 19:58:24.741261 14538 leader_election.cc:290] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2530a39884164942bc417e0905a098f9 (127.10.24.130:39079), 1b540f5979d7433cba2899c26c907c6c (127.10.24.131:33049)
I20250904 19:58:24.753250 14538 raft_consensus.cc:491] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:24.753676 14538 raft_consensus.cc:513] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } }
I20250904 19:58:24.755508 14538 leader_election.cc:290] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2530a39884164942bc417e0905a098f9 (127.10.24.130:39079), 1b540f5979d7433cba2899c26c907c6c (127.10.24.131:33049)
I20250904 19:58:24.764110 14392 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3c4ea89dcf084fba8a4f5f8e73cdfdc4" candidate_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2530a39884164942bc417e0905a098f9" is_pre_election: true
I20250904 19:58:24.764811 14392 raft_consensus.cc:2466] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 2530a39884164942bc417e0905a098f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e5b52e72bcc448dbd6cc93d8fcf8532 in term 0.
I20250904 19:58:24.765750 14481 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3c4ea89dcf084fba8a4f5f8e73cdfdc4" candidate_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b540f5979d7433cba2899c26c907c6c" is_pre_election: true
I20250904 19:58:24.765924 14391 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "dea61aee32184c449f468ef09dbfd1ec" candidate_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2530a39884164942bc417e0905a098f9" is_pre_election: true
I20250904 19:58:24.766438 14481 raft_consensus.cc:2466] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 1b540f5979d7433cba2899c26c907c6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e5b52e72bcc448dbd6cc93d8fcf8532 in term 0.
I20250904 19:58:24.767668 14480 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "dea61aee32184c449f468ef09dbfd1ec" candidate_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b540f5979d7433cba2899c26c907c6c" is_pre_election: true
I20250904 19:58:24.768234 14480 raft_consensus.cc:2466] T dea61aee32184c449f468ef09dbfd1ec P 1b540f5979d7433cba2899c26c907c6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e5b52e72bcc448dbd6cc93d8fcf8532 in term 0.
I20250904 19:58:24.766489 14391 raft_consensus.cc:2466] T dea61aee32184c449f468ef09dbfd1ec P 2530a39884164942bc417e0905a098f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9e5b52e72bcc448dbd6cc93d8fcf8532 in term 0.
I20250904 19:58:24.773303 14279 leader_election.cc:304] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2530a39884164942bc417e0905a098f9, 9e5b52e72bcc448dbd6cc93d8fcf8532; no voters:
I20250904 19:58:24.774012 14279 leader_election.cc:304] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2530a39884164942bc417e0905a098f9, 9e5b52e72bcc448dbd6cc93d8fcf8532; no voters:
I20250904 19:58:24.774509 14538 raft_consensus.cc:2802] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:24.774870 14538 raft_consensus.cc:491] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:24.775213 14538 raft_consensus.cc:3058] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:24.775632 14544 raft_consensus.cc:2802] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:24.776011 14544 raft_consensus.cc:491] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:24.776324 14544 raft_consensus.cc:3058] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:24.783875 14538 raft_consensus.cc:513] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } }
I20250904 19:58:24.786698 14391 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3c4ea89dcf084fba8a4f5f8e73cdfdc4" candidate_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2530a39884164942bc417e0905a098f9"
I20250904 19:58:24.787273 14391 raft_consensus.cc:3058] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 2530a39884164942bc417e0905a098f9 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:24.788157 14480 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3c4ea89dcf084fba8a4f5f8e73cdfdc4" candidate_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b540f5979d7433cba2899c26c907c6c"
I20250904 19:58:24.788704 14480 raft_consensus.cc:3058] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 1b540f5979d7433cba2899c26c907c6c [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:24.795770 14391 raft_consensus.cc:2466] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 2530a39884164942bc417e0905a098f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e5b52e72bcc448dbd6cc93d8fcf8532 in term 1.
I20250904 19:58:24.796779 14279 leader_election.cc:304] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2530a39884164942bc417e0905a098f9, 9e5b52e72bcc448dbd6cc93d8fcf8532; no voters:
I20250904 19:58:24.799593 14480 raft_consensus.cc:2466] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 1b540f5979d7433cba2899c26c907c6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e5b52e72bcc448dbd6cc93d8fcf8532 in term 1.
I20250904 19:58:24.799937 14545 raft_consensus.cc:2802] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:24.800392 14538 leader_election.cc:290] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [CANDIDATE]: Term 1 election: Requested vote from peers 2530a39884164942bc417e0905a098f9 (127.10.24.130:39079), 1b540f5979d7433cba2899c26c907c6c (127.10.24.131:33049)
I20250904 19:58:24.802692 14545 raft_consensus.cc:695] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 1 LEADER]: Becoming Leader. State: Replica: 9e5b52e72bcc448dbd6cc93d8fcf8532, State: Running, Role: LEADER
I20250904 19:58:24.803583 14545 consensus_queue.cc:237] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } }
I20250904 19:58:24.805095 14544 raft_consensus.cc:513] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } }
I20250904 19:58:24.808485 14544 leader_election.cc:290] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [CANDIDATE]: Term 1 election: Requested vote from peers 2530a39884164942bc417e0905a098f9 (127.10.24.130:39079), 1b540f5979d7433cba2899c26c907c6c (127.10.24.131:33049)
I20250904 19:58:24.809466 14480 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "dea61aee32184c449f468ef09dbfd1ec" candidate_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b540f5979d7433cba2899c26c907c6c"
I20250904 19:58:24.810827 14480 raft_consensus.cc:3058] T dea61aee32184c449f468ef09dbfd1ec P 1b540f5979d7433cba2899c26c907c6c [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:24.809716 14391 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "dea61aee32184c449f468ef09dbfd1ec" candidate_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2530a39884164942bc417e0905a098f9"
I20250904 19:58:24.815809 14391 raft_consensus.cc:3058] T dea61aee32184c449f468ef09dbfd1ec P 2530a39884164942bc417e0905a098f9 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:24.817446 14187 catalog_manager.cc:5582] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e5b52e72bcc448dbd6cc93d8fcf8532 (127.10.24.129). New cstate: current_term: 1 leader_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } health_report { overall_health: UNKNOWN } } }
I20250904 19:58:24.825821 14480 raft_consensus.cc:2466] T dea61aee32184c449f468ef09dbfd1ec P 1b540f5979d7433cba2899c26c907c6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e5b52e72bcc448dbd6cc93d8fcf8532 in term 1.
I20250904 19:58:24.829483 14276 leader_election.cc:304] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b540f5979d7433cba2899c26c907c6c, 9e5b52e72bcc448dbd6cc93d8fcf8532; no voters:
I20250904 19:58:24.831023 14545 raft_consensus.cc:2802] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:24.831876 14545 raft_consensus.cc:695] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 1 LEADER]: Becoming Leader. State: Replica: 9e5b52e72bcc448dbd6cc93d8fcf8532, State: Running, Role: LEADER
I20250904 19:58:24.833094 14545 consensus_queue.cc:237] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } }
I20250904 19:58:24.831895 14391 raft_consensus.cc:2466] T dea61aee32184c449f468ef09dbfd1ec P 2530a39884164942bc417e0905a098f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9e5b52e72bcc448dbd6cc93d8fcf8532 in term 1.
I20250904 19:58:24.857668 14189 catalog_manager.cc:5582] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9e5b52e72bcc448dbd6cc93d8fcf8532 (127.10.24.129). New cstate: current_term: 1 leader_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } health_report { overall_health: UNKNOWN } } }
I20250904 19:58:25.007071 14535 raft_consensus.cc:491] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:25.007612 14535 raft_consensus.cc:513] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } }
I20250904 19:58:25.009976 14535 leader_election.cc:290] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e5b52e72bcc448dbd6cc93d8fcf8532 (127.10.24.129:36517), 1b540f5979d7433cba2899c26c907c6c (127.10.24.131:33049)
I20250904 19:58:25.014685 14531 raft_consensus.cc:491] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:25.015220 14531 raft_consensus.cc:513] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } }
I20250904 19:58:25.030076 14531 leader_election.cc:290] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9e5b52e72bcc448dbd6cc93d8fcf8532 (127.10.24.129:36517), 2530a39884164942bc417e0905a098f9 (127.10.24.130:39079)
I20250904 19:58:25.033178 14313 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "334c941721634f9c85c1e0b6a09f4611" candidate_uuid: "2530a39884164942bc417e0905a098f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" is_pre_election: true
I20250904 19:58:25.033898 14313 raft_consensus.cc:2466] T 334c941721634f9c85c1e0b6a09f4611 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2530a39884164942bc417e0905a098f9 in term 0.
I20250904 19:58:25.035226 14355 leader_election.cc:304] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2530a39884164942bc417e0905a098f9, 9e5b52e72bcc448dbd6cc93d8fcf8532; no voters:
I20250904 19:58:25.036098 14535 raft_consensus.cc:2802] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:25.036504 14535 raft_consensus.cc:491] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:25.037202 14535 raft_consensus.cc:3058] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:25.040594 14480 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "334c941721634f9c85c1e0b6a09f4611" candidate_uuid: "2530a39884164942bc417e0905a098f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b540f5979d7433cba2899c26c907c6c" is_pre_election: true
I20250904 19:58:25.041241 14480 raft_consensus.cc:2466] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2530a39884164942bc417e0905a098f9 in term 0.
I20250904 19:58:25.063019 14391 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "334c941721634f9c85c1e0b6a09f4611" candidate_uuid: "1b540f5979d7433cba2899c26c907c6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2530a39884164942bc417e0905a098f9" is_pre_election: true
I20250904 19:58:25.065151 14313 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "334c941721634f9c85c1e0b6a09f4611" candidate_uuid: "1b540f5979d7433cba2899c26c907c6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" is_pre_election: true
I20250904 19:58:25.065809 14313 raft_consensus.cc:2466] T 334c941721634f9c85c1e0b6a09f4611 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b540f5979d7433cba2899c26c907c6c in term 0.
I20250904 19:58:25.066897 14445 leader_election.cc:304] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b540f5979d7433cba2899c26c907c6c, 9e5b52e72bcc448dbd6cc93d8fcf8532; no voters:
I20250904 19:58:25.067764 14531 raft_consensus.cc:2802] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:25.070221 14535 raft_consensus.cc:513] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } }
I20250904 19:58:25.070605 14531 raft_consensus.cc:491] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:25.071105 14531 raft_consensus.cc:3058] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:25.071369 14391 raft_consensus.cc:2391] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b540f5979d7433cba2899c26c907c6c in current term 1: Already voted for candidate 2530a39884164942bc417e0905a098f9 in this term.
I20250904 19:58:25.075474 14535 leader_election.cc:290] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [CANDIDATE]: Term 1 election: Requested vote from peers 9e5b52e72bcc448dbd6cc93d8fcf8532 (127.10.24.129:36517), 1b540f5979d7433cba2899c26c907c6c (127.10.24.131:33049)
I20250904 19:58:25.076282 14313 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "334c941721634f9c85c1e0b6a09f4611" candidate_uuid: "2530a39884164942bc417e0905a098f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532"
I20250904 19:58:25.076859 14313 raft_consensus.cc:3058] T 334c941721634f9c85c1e0b6a09f4611 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:25.080230 14531 raft_consensus.cc:513] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } }
I20250904 19:58:25.082911 14312 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "334c941721634f9c85c1e0b6a09f4611" candidate_uuid: "1b540f5979d7433cba2899c26c907c6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532"
I20250904 19:58:25.084913 14531 leader_election.cc:290] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c [CANDIDATE]: Term 1 election: Requested vote from peers 9e5b52e72bcc448dbd6cc93d8fcf8532 (127.10.24.129:36517), 2530a39884164942bc417e0905a098f9 (127.10.24.130:39079)
I20250904 19:58:25.085465 14313 raft_consensus.cc:2466] T 334c941721634f9c85c1e0b6a09f4611 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2530a39884164942bc417e0905a098f9 in term 1.
I20250904 19:58:25.086419 14391 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "334c941721634f9c85c1e0b6a09f4611" candidate_uuid: "1b540f5979d7433cba2899c26c907c6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2530a39884164942bc417e0905a098f9"
I20250904 19:58:25.086601 14480 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "334c941721634f9c85c1e0b6a09f4611" candidate_uuid: "2530a39884164942bc417e0905a098f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b540f5979d7433cba2899c26c907c6c"
I20250904 19:58:25.086760 14355 leader_election.cc:304] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2530a39884164942bc417e0905a098f9, 9e5b52e72bcc448dbd6cc93d8fcf8532; no voters:
I20250904 19:58:25.087158 14391 raft_consensus.cc:2391] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 1b540f5979d7433cba2899c26c907c6c in current term 1: Already voted for candidate 2530a39884164942bc417e0905a098f9 in this term.
I20250904 19:58:25.087354 14480 raft_consensus.cc:2391] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2530a39884164942bc417e0905a098f9 in current term 1: Already voted for candidate 1b540f5979d7433cba2899c26c907c6c in this term.
I20250904 19:58:25.088454 14535 raft_consensus.cc:2802] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:25.089035 14447 leader_election.cc:304] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b540f5979d7433cba2899c26c907c6c; no voters: 2530a39884164942bc417e0905a098f9, 9e5b52e72bcc448dbd6cc93d8fcf8532
I20250904 19:58:25.089594 14535 raft_consensus.cc:695] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [term 1 LEADER]: Becoming Leader. State: Replica: 2530a39884164942bc417e0905a098f9, State: Running, Role: LEADER
I20250904 19:58:25.089850 14531 raft_consensus.cc:2747] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250904 19:58:25.090636 14535 consensus_queue.cc:237] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } }
I20250904 19:58:25.104090 14187 catalog_manager.cc:5582] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2530a39884164942bc417e0905a098f9 (127.10.24.130). New cstate: current_term: 1 leader_uuid: "2530a39884164942bc417e0905a098f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } health_report { overall_health: HEALTHY } } }
I20250904 19:58:25.234844 14313 raft_consensus.cc:1273] T 334c941721634f9c85c1e0b6a09f4611 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 1 FOLLOWER]: Refusing update from remote peer 2530a39884164942bc417e0905a098f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:25.235033 14480 raft_consensus.cc:1273] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c [term 1 FOLLOWER]: Refusing update from remote peer 2530a39884164942bc417e0905a098f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:25.238330 14535 consensus_queue.cc:1035] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:25.240334 14535 consensus_queue.cc:1035] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20250904 19:58:25.244459 14544 consensus_queue.cc:1035] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:25.264892 14565 mvcc.cc:204] Tried to move back new op lower bound from 7196737147807133696 to 7196737147263643648. Current Snapshot: MvccSnapshot[applied={T|T < 7196737147807133696}]
I20250904 19:58:25.317545 14544 consensus_queue.cc:1035] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:25.331879 14545 consensus_queue.cc:1035] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:25.372646 14575 consensus_queue.cc:1035] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:29.037071 14379 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250904 19:58:29.104501 14470 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250904 19:58:29.116536 14302 tablet_service.cc:1430] Tablet server has 2 leaders and 0 scanners
W20250904 19:58:29.586673 14089 debug-util.cc:398] Leaking SignalData structure 0x7b08001d2880 after lost signal to thread 10341
W20250904 19:58:30.574498 14220 debug-util.cc:398] Leaking SignalData structure 0x7b08000ce000 after lost signal to thread 10341
W20250904 19:58:30.577210 14220 debug-util.cc:398] Leaking SignalData structure 0x7b08000d4e60 after lost signal to thread 14092
W20250904 19:58:30.579423 14220 debug-util.cc:398] Leaking SignalData structure 0x7b0800162fa0 after lost signal to thread 14157
W20250904 19:58:30.595285 14220 debug-util.cc:398] Leaking SignalData structure 0x7b08000f6820 after lost signal to thread 14223
I20250904 19:58:30.612998 14186 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:54616 (request call id 278) took 1192 ms. Trace:
I20250904 19:58:30.613770 14186 rpcz_store.cc:276] 0904 19:58:29.420773 (+ 0us) service_pool.cc:168] Inserting onto call queue
0904 19:58:30.608899 (+1188126us) service_pool.cc:225] Handling call
0904 19:58:30.612688 (+ 3789us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":0}
I20250904 19:58:30.659348 14628 raft_consensus.cc:491] T dea61aee32184c449f468ef09dbfd1ec P 1b540f5979d7433cba2899c26c907c6c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9e5b52e72bcc448dbd6cc93d8fcf8532)
I20250904 19:58:30.660684 14628 raft_consensus.cc:513] T dea61aee32184c449f468ef09dbfd1ec P 1b540f5979d7433cba2899c26c907c6c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 36517 } } peers { permanent_uuid: "2530a39884164942bc417e0905a098f9" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 39079 } } peers { permanent_uuid: "1b540f5979d7433cba2899c26c907c6c" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33049 } }
I20250904 19:58:30.663764 14628 leader_election.cc:290] T dea61aee32184c449f468ef09dbfd1ec P 1b540f5979d7433cba2899c26c907c6c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9e5b52e72bcc448dbd6cc93d8fcf8532 (127.10.24.129:36517), 2530a39884164942bc417e0905a098f9 (127.10.24.130:39079)
I20250904 19:58:30.678115 14629 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:60232 (ReqId={client: 0758f9a54f4c434c9e6e91f4123f3868, seq_no=148, attempt_no=0}) took 1204 ms. Trace:
I20250904 19:58:30.678701 14629 rpcz_store.cc:276] 0904 19:58:29.473129 (+ 0us) service_pool.cc:168] Inserting onto call queue
0904 19:58:29.474036 (+ 907us) service_pool.cc:225] Handling call
0904 19:58:30.677794 (+1203758us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0904 19:58:29.512573 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 3c4ea89dcf084fba8a4f5f8e73cdfdc4
0904 19:58:29.512925 (+ 352us) write_op.cc:432] Acquiring schema lock in shared mode
0904 19:58:29.512951 (+ 26us) write_op.cc:435] Acquired schema lock
0904 19:58:29.512965 (+ 14us) tablet.cc:660] Decoding operations
0904 19:58:29.513192 (+ 227us) write_op.cc:620] Acquiring the partition lock for write op
0904 19:58:29.513238 (+ 46us) write_op.cc:641] Partition lock acquired for write op
0904 19:58:29.513262 (+ 24us) tablet.cc:683] Acquiring locks for 1 operations
0904 19:58:29.513388 (+ 126us) tablet.cc:699] Row locks acquired
0904 19:58:29.513400 (+ 12us) write_op.cc:260] PREPARE: finished
0904 19:58:29.513553 (+ 153us) write_op.cc:270] Start()
0904 19:58:29.513644 (+ 91us) write_op.cc:276] Timestamp: P: 1757015909513512 usec, L: 0
0904 19:58:29.513661 (+ 17us) op_driver.cc:348] REPLICATION: starting
0904 19:58:29.514207 (+ 546us) log.cc:844] Serialized 302 byte log entry
0904 19:58:30.658978 (+1144771us) op_driver.cc:464] REPLICATION: finished
0904 19:58:30.665236 (+ 6258us) write_op.cc:301] APPLY: starting
0904 19:58:30.665372 (+ 136us) tablet.cc:1365] starting BulkCheckPresence
0904 19:58:30.665487 (+ 115us) tablet.cc:1368] finished BulkCheckPresence
0904 19:58:30.667264 (+ 1777us) tablet.cc:1370] starting ApplyRowOperation cycle
0904 19:58:30.667660 (+ 396us) tablet.cc:1381] finished ApplyRowOperation cycle
0904 19:58:30.668025 (+ 365us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=0
0904 19:58:30.668091 (+ 66us) write_op.cc:312] APPLY: finished
0904 19:58:30.668375 (+ 284us) log.cc:844] Serialized 23 byte log entry
0904 19:58:30.676980 (+ 8605us) write_op.cc:489] Releasing partition, row and schema locks
0904 19:58:30.677084 (+ 104us) write_op.cc:454] Released schema lock
0904 19:58:30.677132 (+ 48us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":5934,"num_ops":1,"prepare.queue_time_us":37297,"prepare.run_cpu_time_us":1982,"prepare.run_wall_time_us":1983,"raft.queue_time_us":995,"raft.run_cpu_time_us":735,"raft.run_wall_time_us":735,"replication_time_us":1144887,"thread_start_us":13625,"threads_started":2,"wal-append.queue_time_us":6933}]]}
I20250904 19:58:30.721030 14390 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "dea61aee32184c449f468ef09dbfd1ec" candidate_uuid: "1b540f5979d7433cba2899c26c907c6c" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: false dest_uuid: "2530a39884164942bc417e0905a098f9" is_pre_election: true
W20250904 19:58:30.802805 14220 debug-util.cc:398] Leaking SignalData structure 0x7b080015dfa0 after lost signal to thread 14337
I20250904 19:58:30.805825 14313 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "dea61aee32184c449f468ef09dbfd1ec" candidate_uuid: "1b540f5979d7433cba2899c26c907c6c" candidate_term: 2 candidate_status { last_received { term: 1 index: 11 } } ignore_live_leader: false dest_uuid: "9e5b52e72bcc448dbd6cc93d8fcf8532" is_pre_election: true
I20250904 19:58:30.808231 14445 leader_election.cc:304] T dea61aee32184c449f468ef09dbfd1ec P 1b540f5979d7433cba2899c26c907c6c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b540f5979d7433cba2899c26c907c6c; no voters: 2530a39884164942bc417e0905a098f9, 9e5b52e72bcc448dbd6cc93d8fcf8532
I20250904 19:58:30.810075 14628 raft_consensus.cc:2747] T dea61aee32184c449f468ef09dbfd1ec P 1b540f5979d7433cba2899c26c907c6c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250904 19:58:30.825876 14220 debug-util.cc:398] Leaking SignalData structure 0x7b0800104ea0 after lost signal to thread 14419
W20250904 19:58:30.827145 14220 debug-util.cc:398] Leaking SignalData structure 0x7b0800230480 after lost signal to thread 14505
W20250904 19:58:30.828008 14220 debug-util.cc:398] Leaking SignalData structure 0x7b08000dee20 after lost signal to thread 14620
I20250904 19:58:30.865401 14392 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54418 (request call id 274) took 1408 ms. Trace:
I20250904 19:58:30.865911 14392 rpcz_store.cc:276] 0904 19:58:29.456928 (+ 0us) service_pool.cc:168] Inserting onto call queue
0904 19:58:30.669274 (+1212346us) service_pool.cc:225] Handling call
0904 19:58:30.801517 (+132243us) spinlock_profiling.cc:243] Waited 1.12 ms on lock 0x7b5800004e98. stack: 00007f4d3aefc473 00007f4d3aefc370 00007f4d3a850574 00007f4d448d2784 00007f4d448d2719 00007f4d3ad6f531 00007f4d3aef0ea1 00007f4d3aef0d52 00007f4d43952c39 00007f4d42bfe000 00007f4d42cc4eab 00007f4d42cc4f31 00007f4d42c87f50 00007f4d42c73199 00007f4d42c72c42 00007f4d3c005b44
0904 19:58:30.801770 (+ 253us) raft_consensus.cc:1469] Updating replica for 1 ops
0904 19:58:30.802521 (+ 751us) raft_consensus.cc:1532] Early marking committed up to index 0
0904 19:58:30.802545 (+ 24us) raft_consensus.cc:1537] Triggering prepare for 1 ops
0904 19:58:30.802950 (+ 405us) write_op.cc:270] Start()
0904 19:58:30.803057 (+ 107us) write_op.cc:276] Timestamp: P: 1757015909405243 usec, L: 0
0904 19:58:30.825153 (+ 22096us) log.cc:844] Serialized 302 byte log entry
0904 19:58:30.848676 (+ 23523us) raft_consensus.cc:1643] Marking committed up to 128
0904 19:58:30.848715 (+ 39us) raft_consensus.cc:1695] Filling consensus response to leader.
0904 19:58:30.848780 (+ 65us) raft_consensus.cc:1666] Waiting on the replicates to finish logging
0904 19:58:30.865069 (+ 16289us) raft_consensus.cc:1683] finished
0904 19:58:30.865091 (+ 22us) raft_consensus.cc:1689] UpdateReplicas() finished
0904 19:58:30.865231 (+ 140us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0904 19:58:30.818938 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 3c4ea89dcf084fba8a4f5f8e73cdfdc4
0904 19:58:30.819369 (+ 431us) write_op.cc:432] Acquiring schema lock in shared mode
0904 19:58:30.819403 (+ 34us) write_op.cc:435] Acquired schema lock
0904 19:58:30.819414 (+ 11us) tablet.cc:660] Decoding operations
0904 19:58:30.819707 (+ 293us) write_op.cc:620] Acquiring the partition lock for write op
0904 19:58:30.819795 (+ 88us) write_op.cc:641] Partition lock acquired for write op
0904 19:58:30.819827 (+ 32us) tablet.cc:683] Acquiring locks for 1 operations
0904 19:58:30.820024 (+ 197us) tablet.cc:699] Row locks acquired
0904 19:58:30.820037 (+ 13us) write_op.cc:260] PREPARE: finished
Metrics: {"spinlock_wait_cycles":2564864,"thread_start_us":23136,"threads_started":1,"wal-append.queue_time_us":22797,"child_traces":[["op",{"num_ops":1,"partition_lock_wait_us":37,"prepare.queue_time_us":15417,"prepare.run_cpu_time_us":1467,"prepare.run_wall_time_us":1466,"thread_start_us":13949,"threads_started":1}]]}
I20250904 19:58:31.179872 14332 rpcz_store.cc:275] Call kudu.server.GenericService.GetFlags from 127.0.0.1:60598 (request call id 6) took 1647 ms. Trace:
I20250904 19:58:31.180387 14332 rpcz_store.cc:276] 0904 19:58:29.531887 (+ 0us) service_pool.cc:168] Inserting onto call queue
0904 19:58:29.532048 (+ 161us) service_pool.cc:225] Handling call
0904 19:58:31.179678 (+1647630us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20250904 19:58:31.212013 14276 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.10.24.190:37167, user_credentials={real_user=slave}} blocked reactor thread for 59581.8us
W20250904 19:58:31.272989 14558 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Checksum -> {remote=127.10.24.131:33049, user_credentials={real_user=slave}} blocked reactor thread for 76141us
I20250904 19:58:31.333563 10338 tablet_server.cc:178] TabletServer@127.10.24.129:0 shutting down...
I20250904 19:58:31.443543 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:31.445951 10338 tablet_replica.cc:331] T 334c941721634f9c85c1e0b6a09f4611 P 9e5b52e72bcc448dbd6cc93d8fcf8532: stopping tablet replica
I20250904 19:58:31.447903 10338 raft_consensus.cc:2241] T 334c941721634f9c85c1e0b6a09f4611 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:31.449694 10338 raft_consensus.cc:2270] T 334c941721634f9c85c1e0b6a09f4611 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:31.454304 10338 tablet_replica.cc:331] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532: stopping tablet replica
I20250904 19:58:31.460876 10338 raft_consensus.cc:2241] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:31.463328 10338 raft_consensus.cc:2270] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:31.468891 10338 tablet_replica.cc:331] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532: stopping tablet replica
I20250904 19:58:31.469681 10338 raft_consensus.cc:2241] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:31.479619 10338 raft_consensus.cc:2270] T dea61aee32184c449f468ef09dbfd1ec P 9e5b52e72bcc448dbd6cc93d8fcf8532 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:31.525315 10338 tablet_server.cc:195] TabletServer@127.10.24.129:0 shutdown complete.
I20250904 19:58:31.558017 10338 tablet_server.cc:178] TabletServer@127.10.24.130:0 shutting down...
I20250904 19:58:31.629320 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:31.630198 10338 tablet_replica.cc:331] T dea61aee32184c449f468ef09dbfd1ec P 2530a39884164942bc417e0905a098f9: stopping tablet replica
I20250904 19:58:31.631002 10338 raft_consensus.cc:2241] T dea61aee32184c449f468ef09dbfd1ec P 2530a39884164942bc417e0905a098f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:31.634321 10338 raft_consensus.cc:2270] T dea61aee32184c449f468ef09dbfd1ec P 2530a39884164942bc417e0905a098f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:31.637440 10338 tablet_replica.cc:331] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 2530a39884164942bc417e0905a098f9: stopping tablet replica
I20250904 19:58:31.638231 10338 raft_consensus.cc:2241] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 2530a39884164942bc417e0905a098f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:31.638918 10338 raft_consensus.cc:2270] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 2530a39884164942bc417e0905a098f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:31.646170 10338 tablet_replica.cc:331] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9: stopping tablet replica
I20250904 19:58:31.646874 10338 raft_consensus.cc:2241] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:31.648042 10338 raft_consensus.cc:2270] T 334c941721634f9c85c1e0b6a09f4611 P 2530a39884164942bc417e0905a098f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:31.690910 10338 tablet_server.cc:195] TabletServer@127.10.24.130:0 shutdown complete.
I20250904 19:58:31.716738 10338 tablet_server.cc:178] TabletServer@127.10.24.131:0 shutting down...
I20250904 19:58:31.767331 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:31.768359 10338 tablet_replica.cc:331] T dea61aee32184c449f468ef09dbfd1ec P 1b540f5979d7433cba2899c26c907c6c: stopping tablet replica
I20250904 19:58:31.769311 10338 raft_consensus.cc:2241] T dea61aee32184c449f468ef09dbfd1ec P 1b540f5979d7433cba2899c26c907c6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:31.770202 10338 raft_consensus.cc:2270] T dea61aee32184c449f468ef09dbfd1ec P 1b540f5979d7433cba2899c26c907c6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:31.773387 10338 tablet_replica.cc:331] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 1b540f5979d7433cba2899c26c907c6c: stopping tablet replica
I20250904 19:58:31.774210 10338 raft_consensus.cc:2241] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 1b540f5979d7433cba2899c26c907c6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:31.774839 10338 raft_consensus.cc:2270] T 3c4ea89dcf084fba8a4f5f8e73cdfdc4 P 1b540f5979d7433cba2899c26c907c6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:31.778192 10338 tablet_replica.cc:331] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c: stopping tablet replica
I20250904 19:58:31.778918 10338 raft_consensus.cc:2241] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:31.779601 10338 raft_consensus.cc:2270] T 334c941721634f9c85c1e0b6a09f4611 P 1b540f5979d7433cba2899c26c907c6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:31.804824 10338 tablet_server.cc:195] TabletServer@127.10.24.131:0 shutdown complete.
I20250904 19:58:31.828061 10338 master.cc:561] Master@127.10.24.190:37167 shutting down...
I20250904 19:58:31.861740 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:31.862496 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:31.862907 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 0b3ae42117f64a1798aa4441b428c33e: stopping tablet replica
W20250904 19:58:32.049660 14172 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.24.190:37167: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20250904 19:58:32.054404 14172 consensus_peers.cc:489] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f -> Peer 0b3ae42117f64a1798aa4441b428c33e (127.10.24.190:37167): Couldn't send request to peer 0b3ae42117f64a1798aa4441b428c33e. Status: Network error: Client connection negotiation failed: client connection to 127.10.24.190:37167: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 19:58:32.436882 14625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "709db88c60f648b0b916f3c7e202305f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b3ae42117f64a1798aa4441b428c33e" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 37167 } } peers { permanent_uuid: "84f62af8f4a042049577f0ae62b44d13" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 37045 } } peers { permanent_uuid: "709db88c60f648b0b916f3c7e202305f" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41447 } } }
I20250904 19:58:32.437856 14625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [sys.catalog]: This master's current role is: LEADER
W20250904 19:58:32.873883 10338 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 14089)
W20250904 19:58:33.874367 10338 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 14089)
W20250904 19:58:34.457487 14172 consensus_peers.cc:489] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f -> Peer 0b3ae42117f64a1798aa4441b428c33e (127.10.24.190:37167): Couldn't send request to peer 0b3ae42117f64a1798aa4441b428c33e. Status: Network error: Client connection negotiation failed: client connection to 127.10.24.190:37167: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250904 19:58:34.874804 10338 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 14089)
I20250904 19:58:34.933382 10338 master.cc:583] Master@127.10.24.190:37167 shutdown complete.
I20250904 19:58:34.948814 10338 master.cc:561] Master@127.10.24.189:37045 shutting down...
I20250904 19:58:34.966495 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:34.967100 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:34.967410 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 84f62af8f4a042049577f0ae62b44d13: stopping tablet replica
I20250904 19:58:34.987223 10338 master.cc:583] Master@127.10.24.189:37045 shutdown complete.
I20250904 19:58:35.001125 10338 master.cc:561] Master@127.10.24.188:41447 shutting down...
I20250904 19:58:35.018134 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:35.018980 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:35.019291 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 709db88c60f648b0b916f3c7e202305f: stopping tablet replica
I20250904 19:58:35.062677 10338 master.cc:583] Master@127.10.24.188:41447 shutdown complete.
[ OK ] RemoteKsckTest.TestChecksumSnapshot (11758 ms)
[ RUN ] RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp
I20250904 19:58:35.099407 10338 test_util.cc:276] Using random seed: 628880635
I20250904 19:58:35.100363 10338 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.24.190:42931,127.10.24.189:41829,127.10.24.188:39765
I20250904 19:58:35.101569 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:35.106688 14650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:35.106716 14651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:35.107589 14653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:35.108760 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:35.109656 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:35.109826 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:35.109966 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015915109950 us; error 0 us; skew 500 ppm
I20250904 19:58:35.110453 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:35.112571 10338 webserver.cc:480] Webserver started at http://127.10.24.190:41287/ using document root <none> and password file <none>
I20250904 19:58:35.113011 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:35.113168 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:35.113395 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:35.114436 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/master-0-root/instance:
uuid: "64f552bce012433895d4ee92c17f363b"
format_stamp: "Formatted at 2025-09-04 19:58:35 on dist-test-slave-f3xj"
I20250904 19:58:35.118451 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250904 19:58:35.121435 14658 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:35.122059 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250904 19:58:35.122318 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/master-0-root
uuid: "64f552bce012433895d4ee92c17f363b"
format_stamp: "Formatted at 2025-09-04 19:58:35 on dist-test-slave-f3xj"
I20250904 19:58:35.122556 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:35.137323 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:35.138425 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:35.174475 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.190:42931
I20250904 19:58:35.174593 14709 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.190:42931 every 8 connection(s)
I20250904 19:58:35.178130 14710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:35.178287 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:35.183260 14712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:35.186026 14713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:35.186156 14715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:35.186821 14710 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:35.188171 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:35.189952 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:35.190145 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:35.190300 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015915190285 us; error 0 us; skew 500 ppm
I20250904 19:58:35.190811 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:35.193126 10338 webserver.cc:480] Webserver started at http://127.10.24.189:44711/ using document root <none> and password file <none>
I20250904 19:58:35.193579 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:35.193743 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:35.193965 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:35.194995 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/master-1-root/instance:
uuid: "99b0fcddcfce462594cf90b9dbc20f0b"
format_stamp: "Formatted at 2025-09-04 19:58:35 on dist-test-slave-f3xj"
I20250904 19:58:35.199249 14710 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:35.200031 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20250904 19:58:35.203346 14723 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:35.204146 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
W20250904 19:58:35.204383 14710 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.189:41829: Network error: Client connection negotiation failed: client connection to 127.10.24.189:41829: connect: Connection refused (error 111)
I20250904 19:58:35.204452 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/master-1-root
uuid: "99b0fcddcfce462594cf90b9dbc20f0b"
format_stamp: "Formatted at 2025-09-04 19:58:35 on dist-test-slave-f3xj"
I20250904 19:58:35.204797 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:35.223829 14710 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } attempt: 1
W20250904 19:58:35.227499 14710 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.189:41829: Network error: Client connection negotiation failed: client connection to 127.10.24.189:41829: connect: Connection refused (error 111)
I20250904 19:58:35.233151 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:35.234282 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:35.270645 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.189:41829
I20250904 19:58:35.270759 14774 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.189:41829 every 8 connection(s)
I20250904 19:58:35.274363 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:35.274516 14775 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:35.278942 14710 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } attempt: 2
W20250904 19:58:35.280159 14777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:35.282977 14775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:58:35.284109 14778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:35.288654 14780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:35.292527 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:35.293478 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:35.293710 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:35.293895 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015915293879 us; error 0 us; skew 500 ppm
I20250904 19:58:35.294514 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:35.298094 14775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:35.298159 10338 webserver.cc:480] Webserver started at http://127.10.24.188:33291/ using document root <none> and password file <none>
I20250904 19:58:35.298893 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:35.299145 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:35.299451 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:35.300633 14710 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:35.300855 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/master-2-root/instance:
uuid: "aa26145c3ed0401c8b1a6151c56f1c16"
format_stamp: "Formatted at 2025-09-04 19:58:35 on dist-test-slave-f3xj"
W20250904 19:58:35.313125 14710 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:39765: Network error: Client connection negotiation failed: client connection to 127.10.24.188:39765: connect: Connection refused (error 111)
I20250904 19:58:35.313797 14775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:35.314317 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.013s user 0.009s sys 0.004s
I20250904 19:58:35.317759 14790 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:35.318552 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
W20250904 19:58:35.318758 14775 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:39765: Network error: Client connection negotiation failed: client connection to 127.10.24.188:39765: connect: Connection refused (error 111)
I20250904 19:58:35.318922 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/master-2-root
uuid: "aa26145c3ed0401c8b1a6151c56f1c16"
format_stamp: "Formatted at 2025-09-04 19:58:35 on dist-test-slave-f3xj"
I20250904 19:58:35.319259 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:35.336650 14710 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } attempt: 1
W20250904 19:58:35.340327 14710 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:39765: Network error: Client connection negotiation failed: client connection to 127.10.24.188:39765: connect: Connection refused (error 111)
I20250904 19:58:35.344766 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:35.345947 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:35.381263 14775 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } attempt: 1
I20250904 19:58:35.383412 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.188:39765
I20250904 19:58:35.383612 14841 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.188:39765 every 8 connection(s)
I20250904 19:58:35.387187 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 19:58:35.388139 14843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:35.395872 14843 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:35.401060 14775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b: Bootstrap starting.
I20250904 19:58:35.406123 14775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:35.406335 14843 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:35.410550 14775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b: No bootstrap required, opened a new log
I20250904 19:58:35.412606 14775 raft_consensus.cc:357] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f552bce012433895d4ee92c17f363b" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } } peers { permanent_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } } peers { permanent_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } }
I20250904 19:58:35.413203 14775 raft_consensus.cc:383] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:35.413479 14775 raft_consensus.cc:738] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99b0fcddcfce462594cf90b9dbc20f0b, State: Initialized, Role: FOLLOWER
I20250904 19:58:35.414122 14775 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f552bce012433895d4ee92c17f363b" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } } peers { permanent_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } } peers { permanent_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } }
I20250904 19:58:35.414706 14710 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } attempt: 2
I20250904 19:58:35.416203 14847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f552bce012433895d4ee92c17f363b" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } } peers { permanent_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } } peers { permanent_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } } }
I20250904 19:58:35.416913 14847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:35.417447 14843 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:35.418346 14775 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:35.437718 14710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b: Bootstrap starting.
W20250904 19:58:35.442283 14859 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:58:35.442565 14859 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250904 19:58:35.444309 14710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:35.444595 14843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16: Bootstrap starting.
I20250904 19:58:35.449319 14710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b: No bootstrap required, opened a new log
I20250904 19:58:35.451054 14843 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:35.452106 14710 raft_consensus.cc:357] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f552bce012433895d4ee92c17f363b" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } } peers { permanent_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } } peers { permanent_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } }
I20250904 19:58:35.452836 14710 raft_consensus.cc:383] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:35.453123 14710 raft_consensus.cc:738] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f552bce012433895d4ee92c17f363b, State: Initialized, Role: FOLLOWER
I20250904 19:58:35.453853 14710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f552bce012433895d4ee92c17f363b" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } } peers { permanent_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } } peers { permanent_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } }
I20250904 19:58:35.455641 14862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f552bce012433895d4ee92c17f363b" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } } peers { permanent_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } } peers { permanent_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } } }
I20250904 19:58:35.456048 14843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16: No bootstrap required, opened a new log
I20250904 19:58:35.456141 14862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:35.457319 14710 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:35.458885 14843 raft_consensus.cc:357] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f552bce012433895d4ee92c17f363b" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } } peers { permanent_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } } peers { permanent_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } }
I20250904 19:58:35.459574 14843 raft_consensus.cc:383] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:35.459856 14843 raft_consensus.cc:738] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa26145c3ed0401c8b1a6151c56f1c16, State: Initialized, Role: FOLLOWER
I20250904 19:58:35.460589 14843 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f552bce012433895d4ee92c17f363b" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } } peers { permanent_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } } peers { permanent_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } }
I20250904 19:58:35.462636 14866 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f552bce012433895d4ee92c17f363b" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } } peers { permanent_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } } peers { permanent_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } } }
I20250904 19:58:35.463333 14866 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:35.464401 14843 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:35.467768 14866 raft_consensus.cc:491] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:35.468214 14866 raft_consensus.cc:513] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f552bce012433895d4ee92c17f363b" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } } peers { permanent_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } } peers { permanent_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } }
I20250904 19:58:35.470221 14866 leader_election.cc:290] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f552bce012433895d4ee92c17f363b (127.10.24.190:42931), 99b0fcddcfce462594cf90b9dbc20f0b (127.10.24.189:41829)
I20250904 19:58:35.471143 14685 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f552bce012433895d4ee92c17f363b" is_pre_election: true
I20250904 19:58:35.471837 14685 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa26145c3ed0401c8b1a6151c56f1c16 in term 0.
I20250904 19:58:35.472054 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20250904 19:58:35.472352 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20250904 19:58:35.471861 14750 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" is_pre_election: true
I20250904 19:58:35.473122 14793 leader_election.cc:304] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 64f552bce012433895d4ee92c17f363b, aa26145c3ed0401c8b1a6151c56f1c16; no voters:
I20250904 19:58:35.473647 14750 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa26145c3ed0401c8b1a6151c56f1c16 in term 0.
I20250904 19:58:35.474084 14866 raft_consensus.cc:2802] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:35.474453 14866 raft_consensus.cc:491] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:35.474762 14866 raft_consensus.cc:3058] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [term 0 FOLLOWER]: Advancing to term 1
W20250904 19:58:35.476068 14876 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:58:35.476433 14876 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250904 19:58:35.482559 14866 raft_consensus.cc:513] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f552bce012433895d4ee92c17f363b" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } } peers { permanent_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } } peers { permanent_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } }
I20250904 19:58:35.485116 14685 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f552bce012433895d4ee92c17f363b"
I20250904 19:58:35.485220 14750 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99b0fcddcfce462594cf90b9dbc20f0b"
I20250904 19:58:35.485667 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:35.485677 14685 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:35.485803 14750 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b [term 0 FOLLOWER]: Advancing to term 1
W20250904 19:58:35.488888 14885 catalog_manager.cc:1560] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:58:35.489213 14885 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250904 19:58:35.492239 14750 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa26145c3ed0401c8b1a6151c56f1c16 in term 1.
I20250904 19:58:35.493095 14685 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa26145c3ed0401c8b1a6151c56f1c16 in term 1.
W20250904 19:58:35.493585 14886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:35.493468 14794 leader_election.cc:304] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99b0fcddcfce462594cf90b9dbc20f0b, aa26145c3ed0401c8b1a6151c56f1c16; no voters:
I20250904 19:58:35.494338 14866 leader_election.cc:290] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [CANDIDATE]: Term 1 election: Requested vote from peers 64f552bce012433895d4ee92c17f363b (127.10.24.190:42931), 99b0fcddcfce462594cf90b9dbc20f0b (127.10.24.189:41829)
I20250904 19:58:35.494894 14866 raft_consensus.cc:2802] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [term 1 FOLLOWER]: Leader election won for term 1
W20250904 19:58:35.495057 14887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:35.495390 14866 raft_consensus.cc:695] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [term 1 LEADER]: Becoming Leader. State: Replica: aa26145c3ed0401c8b1a6151c56f1c16, State: Running, Role: LEADER
I20250904 19:58:35.496362 14866 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f552bce012433895d4ee92c17f363b" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } } peers { permanent_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } } peers { permanent_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } }
W20250904 19:58:35.509769 14890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:35.509995 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:35.509966 14889 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa26145c3ed0401c8b1a6151c56f1c16. Latest consensus state: current_term: 1 leader_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f552bce012433895d4ee92c17f363b" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } } peers { permanent_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } } peers { permanent_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } } }
I20250904 19:58:35.510850 14889 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:35.511039 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:35.511267 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:35.511422 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015915511405 us; error 0 us; skew 500 ppm
I20250904 19:58:35.512027 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:35.512190 14893 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 19:58:35.514545 10338 webserver.cc:480] Webserver started at http://127.10.24.129:42647/ using document root <none> and password file <none>
I20250904 19:58:35.515038 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:35.515216 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:35.515470 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:35.516563 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/ts-0-root/instance:
uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4"
format_stamp: "Formatted at 2025-09-04 19:58:35 on dist-test-slave-f3xj"
I20250904 19:58:35.516763 14893 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 19:58:35.521415 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20250904 19:58:35.524381 14750 raft_consensus.cc:1273] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b [term 1 FOLLOWER]: Refusing update from remote peer aa26145c3ed0401c8b1a6151c56f1c16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:35.524382 14685 raft_consensus.cc:1273] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b [term 1 FOLLOWER]: Refusing update from remote peer aa26145c3ed0401c8b1a6151c56f1c16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:35.525648 14898 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:35.525985 14866 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f552bce012433895d4ee92c17f363b" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:35.526578 10338 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20250904 19:58:35.526628 14889 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:35.526918 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/ts-0-root
uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4"
format_stamp: "Formatted at 2025-09-04 19:58:35 on dist-test-slave-f3xj"
I20250904 19:58:35.527253 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:35.536737 14847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa26145c3ed0401c8b1a6151c56f1c16. Latest consensus state: current_term: 1 leader_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f552bce012433895d4ee92c17f363b" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } } peers { permanent_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } } peers { permanent_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } } }
I20250904 19:58:35.537442 14847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:35.543896 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:35.543853 14862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa26145c3ed0401c8b1a6151c56f1c16. Latest consensus state: current_term: 1 leader_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f552bce012433895d4ee92c17f363b" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } } peers { permanent_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } } peers { permanent_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } } }
I20250904 19:58:35.544780 14862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:35.544816 14893 catalog_manager.cc:1349] Generated new cluster ID: f65f51c364774c0a906184c46bce8f8d
I20250904 19:58:35.544687 14866 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f552bce012433895d4ee92c17f363b" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } } peers { permanent_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } } peers { permanent_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } } }
I20250904 19:58:35.545362 14893 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 19:58:35.545483 14866 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:35.551137 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:35.551297 14847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f552bce012433895d4ee92c17f363b" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } } peers { permanent_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } } peers { permanent_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } } }
I20250904 19:58:35.552074 14847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:35.553870 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:35.559092 14866 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f552bce012433895d4ee92c17f363b" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } } peers { permanent_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } } peers { permanent_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } } }
I20250904 19:58:35.560858 14866 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:35.564975 14862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f552bce012433895d4ee92c17f363b" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 42931 } } peers { permanent_uuid: "99b0fcddcfce462594cf90b9dbc20f0b" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41829 } } peers { permanent_uuid: "aa26145c3ed0401c8b1a6151c56f1c16" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39765 } } }
I20250904 19:58:35.565701 14862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:35.573673 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:35.573916 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:35.574133 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:35.574302 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:35.598472 14893 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 19:58:35.600163 14893 catalog_manager.cc:1506] Loading token signing keys...
I20250904 19:58:35.626592 14893 catalog_manager.cc:5955] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16: Generated new TSK 0
I20250904 19:58:35.627300 14893 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 19:58:35.641173 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.129:39961
I20250904 19:58:35.641292 14969 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.129:39961 every 8 connection(s)
I20250904 19:58:35.661409 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:35.672008 14977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:35.672405 14970 heartbeater.cc:344] Connected to a master server at 127.10.24.189:41829
I20250904 19:58:35.672873 14972 heartbeater.cc:344] Connected to a master server at 127.10.24.190:42931
I20250904 19:58:35.673296 14972 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:35.674173 14972 heartbeater.cc:507] Master 127.10.24.190:42931 requested a full tablet report, sending...
I20250904 19:58:35.672874 14970 heartbeater.cc:461] Registering TS with master...
W20250904 19:58:35.674467 14978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:35.674331 14971 heartbeater.cc:344] Connected to a master server at 127.10.24.188:39765
I20250904 19:58:35.675122 14971 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:35.675345 14970 heartbeater.cc:507] Master 127.10.24.189:41829 requested a full tablet report, sending...
I20250904 19:58:35.676064 14971 heartbeater.cc:507] Master 127.10.24.188:39765 requested a full tablet report, sending...
I20250904 19:58:35.677152 14675 ts_manager.cc:194] Registered new tserver with Master: 71d2dc54f3f84c32ad3a4bbccce79bf4 (127.10.24.129:39961)
I20250904 19:58:35.678220 14740 ts_manager.cc:194] Registered new tserver with Master: 71d2dc54f3f84c32ad3a4bbccce79bf4 (127.10.24.129:39961)
I20250904 19:58:35.678668 14807 ts_manager.cc:194] Registered new tserver with Master: 71d2dc54f3f84c32ad3a4bbccce79bf4 (127.10.24.129:39961)
I20250904 19:58:35.681139 14807 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50190
I20250904 19:58:35.681901 10338 server_base.cc:1047] running on GCE node
W20250904 19:58:35.681881 14980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:35.683024 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:35.683226 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:35.683394 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015915683381 us; error 0 us; skew 500 ppm
I20250904 19:58:35.683898 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:35.686021 10338 webserver.cc:480] Webserver started at http://127.10.24.130:42321/ using document root <none> and password file <none>
I20250904 19:58:35.686486 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:35.686659 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:35.686885 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:35.687927 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/ts-1-root/instance:
uuid: "f5c547e04d354e0297a16afa34d18687"
format_stamp: "Formatted at 2025-09-04 19:58:35 on dist-test-slave-f3xj"
I20250904 19:58:35.692939 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20250904 19:58:35.693645 14971 heartbeater.cc:499] Master 127.10.24.188:39765 was elected leader, sending a full tablet report...
I20250904 19:58:35.696272 14985 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:35.696996 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250904 19:58:35.697252 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/ts-1-root
uuid: "f5c547e04d354e0297a16afa34d18687"
format_stamp: "Formatted at 2025-09-04 19:58:35 on dist-test-slave-f3xj"
I20250904 19:58:35.697479 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:35.708962 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:35.710216 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:35.711400 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:35.713548 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:35.713725 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:35.713938 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:35.714080 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:35.760883 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.130:45139
I20250904 19:58:35.760977 15047 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.130:45139 every 8 connection(s)
I20250904 19:58:35.789407 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:35.801394 15055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:35.803541 15048 heartbeater.cc:344] Connected to a master server at 127.10.24.189:41829
I20250904 19:58:35.804023 15048 heartbeater.cc:461] Registering TS with master...
W20250904 19:58:35.804049 15056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:35.805052 15048 heartbeater.cc:507] Master 127.10.24.189:41829 requested a full tablet report, sending...
I20250904 19:58:35.807791 14740 ts_manager.cc:194] Registered new tserver with Master: f5c547e04d354e0297a16afa34d18687 (127.10.24.130:45139)
W20250904 19:58:35.815317 15059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:35.817735 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:35.818959 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:35.819231 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:35.819458 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015915819440 us; error 0 us; skew 500 ppm
I20250904 19:58:35.819979 15049 heartbeater.cc:344] Connected to a master server at 127.10.24.188:39765
I20250904 19:58:35.820268 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:35.820395 15049 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:35.821581 15049 heartbeater.cc:507] Master 127.10.24.188:39765 requested a full tablet report, sending...
I20250904 19:58:35.823881 14807 ts_manager.cc:194] Registered new tserver with Master: f5c547e04d354e0297a16afa34d18687 (127.10.24.130:45139)
I20250904 19:58:35.825815 14807 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50198
I20250904 19:58:35.833097 10338 webserver.cc:480] Webserver started at http://127.10.24.131:34421/ using document root <none> and password file <none>
I20250904 19:58:35.833352 15050 heartbeater.cc:344] Connected to a master server at 127.10.24.190:42931
I20250904 19:58:35.833704 15050 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:35.833735 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:35.834049 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:35.834311 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:35.834291 15050 heartbeater.cc:507] Master 127.10.24.190:42931 requested a full tablet report, sending...
I20250904 19:58:35.835688 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/ts-2-root/instance:
uuid: "475cb10b0a0f4430984655ea3639ea62"
format_stamp: "Formatted at 2025-09-04 19:58:35 on dist-test-slave-f3xj"
I20250904 19:58:35.836774 14675 ts_manager.cc:194] Registered new tserver with Master: f5c547e04d354e0297a16afa34d18687 (127.10.24.130:45139)
I20250904 19:58:35.838994 15049 heartbeater.cc:499] Master 127.10.24.188:39765 was elected leader, sending a full tablet report...
I20250904 19:58:35.842650 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.000s sys 0.005s
I20250904 19:58:35.846160 15063 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:35.846843 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250904 19:58:35.847091 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/ts-2-root
uuid: "475cb10b0a0f4430984655ea3639ea62"
format_stamp: "Formatted at 2025-09-04 19:58:35 on dist-test-slave-f3xj"
I20250904 19:58:35.847326 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp.1757015869512910-10338-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:35.878028 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:35.879580 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:35.881028 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:35.884125 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:35.884333 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:35.884912 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:35.885053 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:35.957295 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.131:45789
I20250904 19:58:35.957340 15125 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.131:45789 every 8 connection(s)
I20250904 19:58:35.983752 15126 heartbeater.cc:344] Connected to a master server at 127.10.24.189:41829
I20250904 19:58:35.984207 15126 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:35.985411 15126 heartbeater.cc:507] Master 127.10.24.189:41829 requested a full tablet report, sending...
I20250904 19:58:35.988721 14740 ts_manager.cc:194] Registered new tserver with Master: 475cb10b0a0f4430984655ea3639ea62 (127.10.24.131:45789)
I20250904 19:58:35.990680 15127 heartbeater.cc:344] Connected to a master server at 127.10.24.188:39765
I20250904 19:58:35.991082 15127 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:35.991927 15127 heartbeater.cc:507] Master 127.10.24.188:39765 requested a full tablet report, sending...
I20250904 19:58:35.993036 15128 heartbeater.cc:344] Connected to a master server at 127.10.24.190:42931
I20250904 19:58:35.993364 15128 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:35.993896 14806 ts_manager.cc:194] Registered new tserver with Master: 475cb10b0a0f4430984655ea3639ea62 (127.10.24.131:45789)
I20250904 19:58:35.994053 15128 heartbeater.cc:507] Master 127.10.24.190:42931 requested a full tablet report, sending...
I20250904 19:58:35.995777 14806 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50210
I20250904 19:58:35.997239 14675 ts_manager.cc:194] Registered new tserver with Master: 475cb10b0a0f4430984655ea3639ea62 (127.10.24.131:45789)
I20250904 19:58:35.998678 10338 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018916757s
I20250904 19:58:36.011493 15127 heartbeater.cc:499] Master 127.10.24.188:39765 was elected leader, sending a full tablet report...
I20250904 19:58:36.057852 14806 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50224:
name: "ksck-test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\n\000\000\000\004\001\024\000\000\000""\004\001\n\000\000\000\004\001\024\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 19:58:36.060907 14806 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table ksck-test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 19:58:36.180194 14932 tablet_service.cc:1468] Processing CreateTablet for tablet 3a3a53c8fbda4ab1ba31af32b49a9fac (DEFAULT_TABLE table=ksck-test-table [id=19035763443b4e7489573191248d1f16]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:36.181560 14932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a3a53c8fbda4ab1ba31af32b49a9fac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:36.182744 15091 tablet_service.cc:1468] Processing CreateTablet for tablet a975e6cc640a4630923d59ba598405ce (DEFAULT_TABLE table=ksck-test-table [id=19035763443b4e7489573191248d1f16]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:36.183995 15091 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a975e6cc640a4630923d59ba598405ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:36.185215 14934 tablet_service.cc:1468] Processing CreateTablet for tablet a975e6cc640a4630923d59ba598405ce (DEFAULT_TABLE table=ksck-test-table [id=19035763443b4e7489573191248d1f16]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:36.186376 14934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a975e6cc640a4630923d59ba598405ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:36.187824 14933 tablet_service.cc:1468] Processing CreateTablet for tablet 1e2ec164ba94432fbbfb098d97a4e3b6 (DEFAULT_TABLE table=ksck-test-table [id=19035763443b4e7489573191248d1f16]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:36.189039 14933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e2ec164ba94432fbbfb098d97a4e3b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:36.197672 15090 tablet_service.cc:1468] Processing CreateTablet for tablet 1e2ec164ba94432fbbfb098d97a4e3b6 (DEFAULT_TABLE table=ksck-test-table [id=19035763443b4e7489573191248d1f16]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:36.199033 15090 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e2ec164ba94432fbbfb098d97a4e3b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:36.202913 15012 tablet_service.cc:1468] Processing CreateTablet for tablet 1e2ec164ba94432fbbfb098d97a4e3b6 (DEFAULT_TABLE table=ksck-test-table [id=19035763443b4e7489573191248d1f16]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:36.203507 15013 tablet_service.cc:1468] Processing CreateTablet for tablet a975e6cc640a4630923d59ba598405ce (DEFAULT_TABLE table=ksck-test-table [id=19035763443b4e7489573191248d1f16]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:36.204263 15012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e2ec164ba94432fbbfb098d97a4e3b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:36.205192 15089 tablet_service.cc:1468] Processing CreateTablet for tablet 3a3a53c8fbda4ab1ba31af32b49a9fac (DEFAULT_TABLE table=ksck-test-table [id=19035763443b4e7489573191248d1f16]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:36.206446 15089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a3a53c8fbda4ab1ba31af32b49a9fac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:36.210714 15013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a975e6cc640a4630923d59ba598405ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:36.215381 15011 tablet_service.cc:1468] Processing CreateTablet for tablet 3a3a53c8fbda4ab1ba31af32b49a9fac (DEFAULT_TABLE table=ksck-test-table [id=19035763443b4e7489573191248d1f16]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:36.216662 15011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a3a53c8fbda4ab1ba31af32b49a9fac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:36.244177 15147 tablet_bootstrap.cc:492] T 1e2ec164ba94432fbbfb098d97a4e3b6 P f5c547e04d354e0297a16afa34d18687: Bootstrap starting.
I20250904 19:58:36.253052 15147 tablet_bootstrap.cc:654] T 1e2ec164ba94432fbbfb098d97a4e3b6 P f5c547e04d354e0297a16afa34d18687: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:36.265756 15147 tablet_bootstrap.cc:492] T 1e2ec164ba94432fbbfb098d97a4e3b6 P f5c547e04d354e0297a16afa34d18687: No bootstrap required, opened a new log
I20250904 19:58:36.266299 15147 ts_tablet_manager.cc:1397] T 1e2ec164ba94432fbbfb098d97a4e3b6 P f5c547e04d354e0297a16afa34d18687: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.006s
I20250904 19:58:36.267161 15148 tablet_bootstrap.cc:492] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62: Bootstrap starting.
I20250904 19:58:36.270349 15150 tablet_bootstrap.cc:492] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4: Bootstrap starting.
I20250904 19:58:36.273042 15148 tablet_bootstrap.cc:654] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:36.283840 15150 tablet_bootstrap.cc:654] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:36.285358 15147 raft_consensus.cc:357] T 1e2ec164ba94432fbbfb098d97a4e3b6 P f5c547e04d354e0297a16afa34d18687 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } }
I20250904 19:58:36.286108 15147 raft_consensus.cc:383] T 1e2ec164ba94432fbbfb098d97a4e3b6 P f5c547e04d354e0297a16afa34d18687 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:36.286410 15147 raft_consensus.cc:738] T 1e2ec164ba94432fbbfb098d97a4e3b6 P f5c547e04d354e0297a16afa34d18687 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5c547e04d354e0297a16afa34d18687, State: Initialized, Role: FOLLOWER
I20250904 19:58:36.287187 15147 consensus_queue.cc:260] T 1e2ec164ba94432fbbfb098d97a4e3b6 P f5c547e04d354e0297a16afa34d18687 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } }
I20250904 19:58:36.300050 15148 tablet_bootstrap.cc:492] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62: No bootstrap required, opened a new log
I20250904 19:58:36.300591 15148 ts_tablet_manager.cc:1397] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62: Time spent bootstrapping tablet: real 0.034s user 0.016s sys 0.004s
I20250904 19:58:36.304364 15150 tablet_bootstrap.cc:492] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4: No bootstrap required, opened a new log
I20250904 19:58:36.304689 15147 ts_tablet_manager.cc:1428] T 1e2ec164ba94432fbbfb098d97a4e3b6 P f5c547e04d354e0297a16afa34d18687: Time spent starting tablet: real 0.038s user 0.007s sys 0.003s
I20250904 19:58:36.304976 15150 ts_tablet_manager.cc:1397] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4: Time spent bootstrapping tablet: real 0.035s user 0.008s sys 0.010s
I20250904 19:58:36.315980 15148 raft_consensus.cc:357] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } }
I20250904 19:58:36.316723 15148 raft_consensus.cc:383] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:36.317065 15148 raft_consensus.cc:738] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 475cb10b0a0f4430984655ea3639ea62, State: Initialized, Role: FOLLOWER
I20250904 19:58:36.317685 15148 consensus_queue.cc:260] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } }
I20250904 19:58:36.321015 15150 raft_consensus.cc:357] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } }
I20250904 19:58:36.321605 15150 raft_consensus.cc:383] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:36.321848 15150 raft_consensus.cc:738] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71d2dc54f3f84c32ad3a4bbccce79bf4, State: Initialized, Role: FOLLOWER
I20250904 19:58:36.307358 15147 tablet_bootstrap.cc:492] T 3a3a53c8fbda4ab1ba31af32b49a9fac P f5c547e04d354e0297a16afa34d18687: Bootstrap starting.
I20250904 19:58:36.327704 15148 ts_tablet_manager.cc:1428] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62: Time spent starting tablet: real 0.027s user 0.010s sys 0.004s
I20250904 19:58:36.324646 15150 consensus_queue.cc:260] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } }
I20250904 19:58:36.329473 15147 tablet_bootstrap.cc:654] T 3a3a53c8fbda4ab1ba31af32b49a9fac P f5c547e04d354e0297a16afa34d18687: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:36.338827 15147 tablet_bootstrap.cc:492] T 3a3a53c8fbda4ab1ba31af32b49a9fac P f5c547e04d354e0297a16afa34d18687: No bootstrap required, opened a new log
I20250904 19:58:36.339299 15147 ts_tablet_manager.cc:1397] T 3a3a53c8fbda4ab1ba31af32b49a9fac P f5c547e04d354e0297a16afa34d18687: Time spent bootstrapping tablet: real 0.032s user 0.008s sys 0.000s
I20250904 19:58:36.341981 15147 raft_consensus.cc:357] T 3a3a53c8fbda4ab1ba31af32b49a9fac P f5c547e04d354e0297a16afa34d18687 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } }
I20250904 19:58:36.342643 15147 raft_consensus.cc:383] T 3a3a53c8fbda4ab1ba31af32b49a9fac P f5c547e04d354e0297a16afa34d18687 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:36.342917 15147 raft_consensus.cc:738] T 3a3a53c8fbda4ab1ba31af32b49a9fac P f5c547e04d354e0297a16afa34d18687 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5c547e04d354e0297a16afa34d18687, State: Initialized, Role: FOLLOWER
I20250904 19:58:36.343771 15148 tablet_bootstrap.cc:492] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62: Bootstrap starting.
I20250904 19:58:36.354408 15147 consensus_queue.cc:260] T 3a3a53c8fbda4ab1ba31af32b49a9fac P f5c547e04d354e0297a16afa34d18687 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } }
I20250904 19:58:36.358461 15147 ts_tablet_manager.cc:1428] T 3a3a53c8fbda4ab1ba31af32b49a9fac P f5c547e04d354e0297a16afa34d18687: Time spent starting tablet: real 0.019s user 0.004s sys 0.001s
I20250904 19:58:36.361239 15147 tablet_bootstrap.cc:492] T a975e6cc640a4630923d59ba598405ce P f5c547e04d354e0297a16afa34d18687: Bootstrap starting.
I20250904 19:58:36.364066 15148 tablet_bootstrap.cc:654] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:36.366544 15150 ts_tablet_manager.cc:1428] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4: Time spent starting tablet: real 0.057s user 0.009s sys 0.009s
I20250904 19:58:36.367777 15150 tablet_bootstrap.cc:492] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 71d2dc54f3f84c32ad3a4bbccce79bf4: Bootstrap starting.
I20250904 19:58:36.373978 15150 tablet_bootstrap.cc:654] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 71d2dc54f3f84c32ad3a4bbccce79bf4: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:36.374804 15148 tablet_bootstrap.cc:492] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62: No bootstrap required, opened a new log
I20250904 19:58:36.381744 15148 ts_tablet_manager.cc:1397] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62: Time spent bootstrapping tablet: real 0.038s user 0.011s sys 0.002s
I20250904 19:58:36.378659 15147 tablet_bootstrap.cc:654] T a975e6cc640a4630923d59ba598405ce P f5c547e04d354e0297a16afa34d18687: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:36.385488 15148 raft_consensus.cc:357] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } }
I20250904 19:58:36.386718 15148 raft_consensus.cc:383] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:36.386993 15148 raft_consensus.cc:738] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 475cb10b0a0f4430984655ea3639ea62, State: Initialized, Role: FOLLOWER
I20250904 19:58:36.387720 15148 consensus_queue.cc:260] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } }
I20250904 19:58:36.390072 15148 ts_tablet_manager.cc:1428] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62: Time spent starting tablet: real 0.008s user 0.002s sys 0.003s
I20250904 19:58:36.391150 15148 tablet_bootstrap.cc:492] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 475cb10b0a0f4430984655ea3639ea62: Bootstrap starting.
I20250904 19:58:36.399360 15148 tablet_bootstrap.cc:654] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 475cb10b0a0f4430984655ea3639ea62: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:36.408231 15150 tablet_bootstrap.cc:492] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 71d2dc54f3f84c32ad3a4bbccce79bf4: No bootstrap required, opened a new log
I20250904 19:58:36.412627 15150 ts_tablet_manager.cc:1397] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 71d2dc54f3f84c32ad3a4bbccce79bf4: Time spent bootstrapping tablet: real 0.045s user 0.011s sys 0.000s
I20250904 19:58:36.418342 15150 raft_consensus.cc:357] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } }
I20250904 19:58:36.419054 15150 raft_consensus.cc:383] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:36.420791 15147 tablet_bootstrap.cc:492] T a975e6cc640a4630923d59ba598405ce P f5c547e04d354e0297a16afa34d18687: No bootstrap required, opened a new log
I20250904 19:58:36.421586 15148 tablet_bootstrap.cc:492] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 475cb10b0a0f4430984655ea3639ea62: No bootstrap required, opened a new log
I20250904 19:58:36.421679 15147 ts_tablet_manager.cc:1397] T a975e6cc640a4630923d59ba598405ce P f5c547e04d354e0297a16afa34d18687: Time spent bootstrapping tablet: real 0.061s user 0.014s sys 0.010s
I20250904 19:58:36.422111 15148 ts_tablet_manager.cc:1397] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 475cb10b0a0f4430984655ea3639ea62: Time spent bootstrapping tablet: real 0.031s user 0.008s sys 0.000s
I20250904 19:58:36.424748 15147 raft_consensus.cc:357] T a975e6cc640a4630923d59ba598405ce P f5c547e04d354e0297a16afa34d18687 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } }
I20250904 19:58:36.425060 15148 raft_consensus.cc:357] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } }
I20250904 19:58:36.425524 15147 raft_consensus.cc:383] T a975e6cc640a4630923d59ba598405ce P f5c547e04d354e0297a16afa34d18687 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:36.425770 15148 raft_consensus.cc:383] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:36.425866 15147 raft_consensus.cc:738] T a975e6cc640a4630923d59ba598405ce P f5c547e04d354e0297a16afa34d18687 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5c547e04d354e0297a16afa34d18687, State: Initialized, Role: FOLLOWER
I20250904 19:58:36.426215 15148 raft_consensus.cc:738] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 475cb10b0a0f4430984655ea3639ea62, State: Initialized, Role: FOLLOWER
I20250904 19:58:36.426715 15147 consensus_queue.cc:260] T a975e6cc640a4630923d59ba598405ce P f5c547e04d354e0297a16afa34d18687 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } }
I20250904 19:58:36.427063 15148 consensus_queue.cc:260] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 475cb10b0a0f4430984655ea3639ea62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } }
I20250904 19:58:36.429023 15147 ts_tablet_manager.cc:1428] T a975e6cc640a4630923d59ba598405ce P f5c547e04d354e0297a16afa34d18687: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20250904 19:58:36.419366 15150 raft_consensus.cc:738] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71d2dc54f3f84c32ad3a4bbccce79bf4, State: Initialized, Role: FOLLOWER
I20250904 19:58:36.427714 15154 raft_consensus.cc:491] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:36.430935 15150 consensus_queue.cc:260] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 71d2dc54f3f84c32ad3a4bbccce79bf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } }
I20250904 19:58:36.431609 15154 raft_consensus.cc:513] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } }
I20250904 19:58:36.435534 15154 leader_election.cc:290] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71d2dc54f3f84c32ad3a4bbccce79bf4 (127.10.24.129:39961), f5c547e04d354e0297a16afa34d18687 (127.10.24.130:45139)
I20250904 19:58:36.440577 15150 ts_tablet_manager.cc:1428] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 71d2dc54f3f84c32ad3a4bbccce79bf4: Time spent starting tablet: real 0.028s user 0.007s sys 0.000s
I20250904 19:58:36.446864 15150 tablet_bootstrap.cc:492] T a975e6cc640a4630923d59ba598405ce P 71d2dc54f3f84c32ad3a4bbccce79bf4: Bootstrap starting.
I20250904 19:58:36.457672 14944 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1e2ec164ba94432fbbfb098d97a4e3b6" candidate_uuid: "475cb10b0a0f4430984655ea3639ea62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" is_pre_election: true
I20250904 19:58:36.458385 14944 raft_consensus.cc:2466] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 475cb10b0a0f4430984655ea3639ea62 in term 0.
I20250904 19:58:36.459128 15023 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1e2ec164ba94432fbbfb098d97a4e3b6" candidate_uuid: "475cb10b0a0f4430984655ea3639ea62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5c547e04d354e0297a16afa34d18687" is_pre_election: true
I20250904 19:58:36.459807 15023 raft_consensus.cc:2466] T 1e2ec164ba94432fbbfb098d97a4e3b6 P f5c547e04d354e0297a16afa34d18687 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 475cb10b0a0f4430984655ea3639ea62 in term 0.
I20250904 19:58:36.460309 15148 ts_tablet_manager.cc:1428] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 475cb10b0a0f4430984655ea3639ea62: Time spent starting tablet: real 0.038s user 0.013s sys 0.001s
I20250904 19:58:36.460887 15065 leader_election.cc:304] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 475cb10b0a0f4430984655ea3639ea62, 71d2dc54f3f84c32ad3a4bbccce79bf4; no voters:
I20250904 19:58:36.464143 14859 catalog_manager.cc:1261] Loaded cluster ID: f65f51c364774c0a906184c46bce8f8d
I20250904 19:58:36.464422 14859 catalog_manager.cc:1554] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b: loading cluster ID for follower catalog manager: success
I20250904 19:58:36.466338 15150 tablet_bootstrap.cc:654] T a975e6cc640a4630923d59ba598405ce P 71d2dc54f3f84c32ad3a4bbccce79bf4: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:36.474876 15161 raft_consensus.cc:2802] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:36.475445 15161 raft_consensus.cc:491] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:36.475864 15161 raft_consensus.cc:3058] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:36.481575 14859 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b: acquiring CA information for follower catalog manager: success
I20250904 19:58:36.482561 14876 catalog_manager.cc:1261] Loaded cluster ID: f65f51c364774c0a906184c46bce8f8d
I20250904 19:58:36.482846 14876 catalog_manager.cc:1554] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b: loading cluster ID for follower catalog manager: success
I20250904 19:58:36.486199 14859 catalog_manager.cc:1604] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:58:36.488339 15161 raft_consensus.cc:513] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } }
I20250904 19:58:36.490371 15161 leader_election.cc:290] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [CANDIDATE]: Term 1 election: Requested vote from peers 71d2dc54f3f84c32ad3a4bbccce79bf4 (127.10.24.129:39961), f5c547e04d354e0297a16afa34d18687 (127.10.24.130:45139)
I20250904 19:58:36.491410 15023 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1e2ec164ba94432fbbfb098d97a4e3b6" candidate_uuid: "475cb10b0a0f4430984655ea3639ea62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5c547e04d354e0297a16afa34d18687"
I20250904 19:58:36.492013 15023 raft_consensus.cc:3058] T 1e2ec164ba94432fbbfb098d97a4e3b6 P f5c547e04d354e0297a16afa34d18687 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:36.495245 14876 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b: acquiring CA information for follower catalog manager: success
I20250904 19:58:36.495862 14944 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1e2ec164ba94432fbbfb098d97a4e3b6" candidate_uuid: "475cb10b0a0f4430984655ea3639ea62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4"
I20250904 19:58:36.498718 14944 raft_consensus.cc:3058] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:36.501966 15023 raft_consensus.cc:2466] T 1e2ec164ba94432fbbfb098d97a4e3b6 P f5c547e04d354e0297a16afa34d18687 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 475cb10b0a0f4430984655ea3639ea62 in term 1.
I20250904 19:58:36.503121 15067 leader_election.cc:304] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 475cb10b0a0f4430984655ea3639ea62, f5c547e04d354e0297a16afa34d18687; no voters:
I20250904 19:58:36.504062 15161 raft_consensus.cc:2802] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:36.505271 15161 raft_consensus.cc:695] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [term 1 LEADER]: Becoming Leader. State: Replica: 475cb10b0a0f4430984655ea3639ea62, State: Running, Role: LEADER
I20250904 19:58:36.505672 15150 tablet_bootstrap.cc:492] T a975e6cc640a4630923d59ba598405ce P 71d2dc54f3f84c32ad3a4bbccce79bf4: No bootstrap required, opened a new log
I20250904 19:58:36.506060 15161 consensus_queue.cc:237] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } }
I20250904 19:58:36.508113 14944 raft_consensus.cc:2466] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 475cb10b0a0f4430984655ea3639ea62 in term 1.
I20250904 19:58:36.506150 15150 ts_tablet_manager.cc:1397] T a975e6cc640a4630923d59ba598405ce P 71d2dc54f3f84c32ad3a4bbccce79bf4: Time spent bootstrapping tablet: real 0.059s user 0.015s sys 0.005s
I20250904 19:58:36.518612 15150 raft_consensus.cc:357] T a975e6cc640a4630923d59ba598405ce P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } }
I20250904 19:58:36.519321 15150 raft_consensus.cc:383] T a975e6cc640a4630923d59ba598405ce P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:36.520848 14876 catalog_manager.cc:1604] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:58:36.521039 15150 raft_consensus.cc:738] T a975e6cc640a4630923d59ba598405ce P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71d2dc54f3f84c32ad3a4bbccce79bf4, State: Initialized, Role: FOLLOWER
I20250904 19:58:36.521838 15150 consensus_queue.cc:260] T a975e6cc640a4630923d59ba598405ce P 71d2dc54f3f84c32ad3a4bbccce79bf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } }
I20250904 19:58:36.524288 15150 ts_tablet_manager.cc:1428] T a975e6cc640a4630923d59ba598405ce P 71d2dc54f3f84c32ad3a4bbccce79bf4: Time spent starting tablet: real 0.008s user 0.004s sys 0.003s
I20250904 19:58:36.525063 15155 raft_consensus.cc:491] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:36.525516 15155 raft_consensus.cc:513] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } }
I20250904 19:58:36.527949 15155 leader_election.cc:290] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5c547e04d354e0297a16afa34d18687 (127.10.24.130:45139), 475cb10b0a0f4430984655ea3639ea62 (127.10.24.131:45789)
I20250904 19:58:36.546991 14807 catalog_manager.cc:5582] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 reported cstate change: term changed from 0 to 1, leader changed from <none> to 475cb10b0a0f4430984655ea3639ea62 (127.10.24.131). New cstate: current_term: 1 leader_uuid: "475cb10b0a0f4430984655ea3639ea62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } health_report { overall_health: HEALTHY } } }
I20250904 19:58:36.564635 15023 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3a3a53c8fbda4ab1ba31af32b49a9fac" candidate_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5c547e04d354e0297a16afa34d18687" is_pre_election: true
I20250904 19:58:36.565289 15023 raft_consensus.cc:2466] T 3a3a53c8fbda4ab1ba31af32b49a9fac P f5c547e04d354e0297a16afa34d18687 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71d2dc54f3f84c32ad3a4bbccce79bf4 in term 0.
I20250904 19:58:36.570013 14904 leader_election.cc:304] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71d2dc54f3f84c32ad3a4bbccce79bf4, f5c547e04d354e0297a16afa34d18687; no voters:
I20250904 19:58:36.571727 15155 raft_consensus.cc:2802] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:36.572084 15155 raft_consensus.cc:491] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:36.572427 15155 raft_consensus.cc:3058] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:36.583518 15101 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3a3a53c8fbda4ab1ba31af32b49a9fac" candidate_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "475cb10b0a0f4430984655ea3639ea62" is_pre_election: true
I20250904 19:58:36.583745 15155 raft_consensus.cc:513] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } }
I20250904 19:58:36.584211 15101 raft_consensus.cc:2466] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71d2dc54f3f84c32ad3a4bbccce79bf4 in term 0.
I20250904 19:58:36.586552 15155 leader_election.cc:290] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [CANDIDATE]: Term 1 election: Requested vote from peers f5c547e04d354e0297a16afa34d18687 (127.10.24.130:45139), 475cb10b0a0f4430984655ea3639ea62 (127.10.24.131:45789)
I20250904 19:58:36.587169 15101 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3a3a53c8fbda4ab1ba31af32b49a9fac" candidate_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "475cb10b0a0f4430984655ea3639ea62"
I20250904 19:58:36.587369 15023 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3a3a53c8fbda4ab1ba31af32b49a9fac" candidate_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5c547e04d354e0297a16afa34d18687"
I20250904 19:58:36.587744 15101 raft_consensus.cc:3058] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:36.590509 15023 raft_consensus.cc:3058] T 3a3a53c8fbda4ab1ba31af32b49a9fac P f5c547e04d354e0297a16afa34d18687 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:36.597779 15023 raft_consensus.cc:2466] T 3a3a53c8fbda4ab1ba31af32b49a9fac P f5c547e04d354e0297a16afa34d18687 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71d2dc54f3f84c32ad3a4bbccce79bf4 in term 1.
I20250904 19:58:36.599021 14904 leader_election.cc:304] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71d2dc54f3f84c32ad3a4bbccce79bf4, f5c547e04d354e0297a16afa34d18687; no voters:
I20250904 19:58:36.600131 15155 raft_consensus.cc:2802] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:36.600921 15101 raft_consensus.cc:2466] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 475cb10b0a0f4430984655ea3639ea62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71d2dc54f3f84c32ad3a4bbccce79bf4 in term 1.
I20250904 19:58:36.601106 15155 raft_consensus.cc:695] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 1 LEADER]: Becoming Leader. State: Replica: 71d2dc54f3f84c32ad3a4bbccce79bf4, State: Running, Role: LEADER
I20250904 19:58:36.602017 15155 consensus_queue.cc:237] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } }
I20250904 19:58:36.613077 14805 catalog_manager.cc:5582] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 71d2dc54f3f84c32ad3a4bbccce79bf4 (127.10.24.129). New cstate: current_term: 1 leader_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } health_report { overall_health: UNKNOWN } } }
I20250904 19:58:36.748819 15161 raft_consensus.cc:491] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:36.749352 15161 raft_consensus.cc:513] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } }
I20250904 19:58:36.751324 15161 leader_election.cc:290] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71d2dc54f3f84c32ad3a4bbccce79bf4 (127.10.24.129:39961), f5c547e04d354e0297a16afa34d18687 (127.10.24.130:45139)
I20250904 19:58:36.752759 14944 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a975e6cc640a4630923d59ba598405ce" candidate_uuid: "475cb10b0a0f4430984655ea3639ea62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" is_pre_election: true
I20250904 19:58:36.753119 15023 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a975e6cc640a4630923d59ba598405ce" candidate_uuid: "475cb10b0a0f4430984655ea3639ea62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5c547e04d354e0297a16afa34d18687" is_pre_election: true
I20250904 19:58:36.753410 14944 raft_consensus.cc:2466] T a975e6cc640a4630923d59ba598405ce P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 475cb10b0a0f4430984655ea3639ea62 in term 0.
I20250904 19:58:36.754055 15023 raft_consensus.cc:2466] T a975e6cc640a4630923d59ba598405ce P f5c547e04d354e0297a16afa34d18687 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 475cb10b0a0f4430984655ea3639ea62 in term 0.
I20250904 19:58:36.754552 15065 leader_election.cc:304] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 475cb10b0a0f4430984655ea3639ea62, 71d2dc54f3f84c32ad3a4bbccce79bf4; no voters:
I20250904 19:58:36.755337 15161 raft_consensus.cc:2802] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:36.755708 15161 raft_consensus.cc:491] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:36.756031 15161 raft_consensus.cc:3058] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:36.762859 15161 raft_consensus.cc:513] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } }
I20250904 19:58:36.765106 14944 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a975e6cc640a4630923d59ba598405ce" candidate_uuid: "475cb10b0a0f4430984655ea3639ea62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4"
I20250904 19:58:36.765547 15023 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a975e6cc640a4630923d59ba598405ce" candidate_uuid: "475cb10b0a0f4430984655ea3639ea62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5c547e04d354e0297a16afa34d18687"
I20250904 19:58:36.765698 14944 raft_consensus.cc:3058] T a975e6cc640a4630923d59ba598405ce P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:36.766127 15023 raft_consensus.cc:3058] T a975e6cc640a4630923d59ba598405ce P f5c547e04d354e0297a16afa34d18687 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:36.764235 15161 leader_election.cc:290] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [CANDIDATE]: Term 1 election: Requested vote from peers 71d2dc54f3f84c32ad3a4bbccce79bf4 (127.10.24.129:39961), f5c547e04d354e0297a16afa34d18687 (127.10.24.130:45139)
I20250904 19:58:36.774495 15023 raft_consensus.cc:2466] T a975e6cc640a4630923d59ba598405ce P f5c547e04d354e0297a16afa34d18687 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 475cb10b0a0f4430984655ea3639ea62 in term 1.
I20250904 19:58:36.775761 15067 leader_election.cc:304] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 475cb10b0a0f4430984655ea3639ea62, f5c547e04d354e0297a16afa34d18687; no voters:
I20250904 19:58:36.776397 14944 raft_consensus.cc:2466] T a975e6cc640a4630923d59ba598405ce P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 475cb10b0a0f4430984655ea3639ea62 in term 1.
I20250904 19:58:36.777315 15161 raft_consensus.cc:2802] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:36.778658 15161 raft_consensus.cc:695] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [term 1 LEADER]: Becoming Leader. State: Replica: 475cb10b0a0f4430984655ea3639ea62, State: Running, Role: LEADER
I20250904 19:58:36.779412 15161 consensus_queue.cc:237] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } }
I20250904 19:58:36.789710 14805 catalog_manager.cc:5582] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 reported cstate change: term changed from 0 to 1, leader changed from <none> to 475cb10b0a0f4430984655ea3639ea62 (127.10.24.131). New cstate: current_term: 1 leader_uuid: "475cb10b0a0f4430984655ea3639ea62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 } health_report { overall_health: UNKNOWN } } }
I20250904 19:58:36.961753 15161 consensus_queue.cc:1035] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:36.983675 15154 consensus_queue.cc:1035] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:37.024560 15023 raft_consensus.cc:1273] T a975e6cc640a4630923d59ba598405ce P f5c547e04d354e0297a16afa34d18687 [term 1 FOLLOWER]: Refusing update from remote peer 475cb10b0a0f4430984655ea3639ea62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:37.026461 15161 consensus_queue.cc:1035] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:37.029716 14943 raft_consensus.cc:1273] T a975e6cc640a4630923d59ba598405ce P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 1 FOLLOWER]: Refusing update from remote peer 475cb10b0a0f4430984655ea3639ea62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:37.031376 15154 consensus_queue.cc:1035] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "71d2dc54f3f84c32ad3a4bbccce79bf4" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:37.129431 15155 consensus_queue.cc:1035] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "475cb10b0a0f4430984655ea3639ea62" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 45789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:37.185488 15155 consensus_queue.cc:1035] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5c547e04d354e0297a16afa34d18687" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 45139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250904 19:58:39.842131 14904 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.24.130:45139, user_credentials={real_user=slave}} blocked reactor thread for 46757.1us
I20250904 19:58:41.105171 15013 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250904 19:58:41.151051 14933 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250904 19:58:41.208559 15091 tablet_service.cc:1430] Tablet server has 2 leaders and 0 scanners
W20250904 19:58:41.861619 15169 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Checksum -> {remote=127.10.24.131:45789, user_credentials={real_user=slave}} blocked reactor thread for 59099.2us
I20250904 19:58:41.895514 10338 tablet_server.cc:178] TabletServer@127.10.24.129:0 shutting down...
I20250904 19:58:41.959440 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:41.960249 10338 tablet_replica.cc:331] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 71d2dc54f3f84c32ad3a4bbccce79bf4: stopping tablet replica
I20250904 19:58:41.960901 10338 raft_consensus.cc:2241] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:41.961413 10338 raft_consensus.cc:2270] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:41.963829 10338 tablet_replica.cc:331] T a975e6cc640a4630923d59ba598405ce P 71d2dc54f3f84c32ad3a4bbccce79bf4: stopping tablet replica
I20250904 19:58:41.965325 10338 raft_consensus.cc:2241] T a975e6cc640a4630923d59ba598405ce P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:41.965984 10338 raft_consensus.cc:2270] T a975e6cc640a4630923d59ba598405ce P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:41.969287 10338 tablet_replica.cc:331] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4: stopping tablet replica
I20250904 19:58:41.969805 10338 raft_consensus.cc:2241] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:41.970732 10338 raft_consensus.cc:2270] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 71d2dc54f3f84c32ad3a4bbccce79bf4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:41.990996 10338 tablet_server.cc:195] TabletServer@127.10.24.129:0 shutdown complete.
W20250904 19:58:41.994217 15065 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.24.129:39961: connect: Connection refused (error 111) [suppressed 12 similar messages]
W20250904 19:58:42.000785 15065 consensus_peers.cc:489] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 -> Peer 71d2dc54f3f84c32ad3a4bbccce79bf4 (127.10.24.129:39961): Couldn't send request to peer 71d2dc54f3f84c32ad3a4bbccce79bf4. Status: Network error: Client connection negotiation failed: client connection to 127.10.24.129:39961: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 19:58:42.015024 10338 tablet_server.cc:178] TabletServer@127.10.24.130:0 shutting down...
I20250904 19:58:42.058167 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:42.058820 10338 tablet_replica.cc:331] T a975e6cc640a4630923d59ba598405ce P f5c547e04d354e0297a16afa34d18687: stopping tablet replica
I20250904 19:58:42.059623 10338 raft_consensus.cc:2241] T a975e6cc640a4630923d59ba598405ce P f5c547e04d354e0297a16afa34d18687 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:42.060319 10338 raft_consensus.cc:2270] T a975e6cc640a4630923d59ba598405ce P f5c547e04d354e0297a16afa34d18687 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:42.062181 10338 tablet_replica.cc:331] T 3a3a53c8fbda4ab1ba31af32b49a9fac P f5c547e04d354e0297a16afa34d18687: stopping tablet replica
I20250904 19:58:42.062695 10338 raft_consensus.cc:2241] T 3a3a53c8fbda4ab1ba31af32b49a9fac P f5c547e04d354e0297a16afa34d18687 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:42.063139 10338 raft_consensus.cc:2270] T 3a3a53c8fbda4ab1ba31af32b49a9fac P f5c547e04d354e0297a16afa34d18687 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:42.065966 10338 tablet_replica.cc:331] T 1e2ec164ba94432fbbfb098d97a4e3b6 P f5c547e04d354e0297a16afa34d18687: stopping tablet replica
I20250904 19:58:42.066529 10338 raft_consensus.cc:2241] T 1e2ec164ba94432fbbfb098d97a4e3b6 P f5c547e04d354e0297a16afa34d18687 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:42.067130 10338 raft_consensus.cc:2270] T 1e2ec164ba94432fbbfb098d97a4e3b6 P f5c547e04d354e0297a16afa34d18687 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:42.087926 10338 tablet_server.cc:195] TabletServer@127.10.24.130:0 shutdown complete.
I20250904 19:58:42.104981 10338 tablet_server.cc:178] TabletServer@127.10.24.131:0 shutting down...
I20250904 19:58:42.139514 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:42.140168 10338 tablet_replica.cc:331] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62: stopping tablet replica
I20250904 19:58:42.140846 10338 raft_consensus.cc:2241] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:42.141847 10338 raft_consensus.cc:2270] T 1e2ec164ba94432fbbfb098d97a4e3b6 P 475cb10b0a0f4430984655ea3639ea62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:42.143776 10338 tablet_replica.cc:331] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 475cb10b0a0f4430984655ea3639ea62: stopping tablet replica
I20250904 19:58:42.144268 10338 raft_consensus.cc:2241] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 475cb10b0a0f4430984655ea3639ea62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:42.144649 10338 raft_consensus.cc:2270] T 3a3a53c8fbda4ab1ba31af32b49a9fac P 475cb10b0a0f4430984655ea3639ea62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:42.146811 10338 tablet_replica.cc:331] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62: stopping tablet replica
I20250904 19:58:42.147250 10338 raft_consensus.cc:2241] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:42.147981 10338 raft_consensus.cc:2270] T a975e6cc640a4630923d59ba598405ce P 475cb10b0a0f4430984655ea3639ea62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:42.170061 10338 tablet_server.cc:195] TabletServer@127.10.24.131:0 shutdown complete.
I20250904 19:58:42.184276 10338 master.cc:561] Master@127.10.24.190:42931 shutting down...
I20250904 19:58:42.206157 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:42.206686 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:42.207098 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 64f552bce012433895d4ee92c17f363b: stopping tablet replica
I20250904 19:58:42.226295 10338 master.cc:583] Master@127.10.24.190:42931 shutdown complete.
I20250904 19:58:42.241798 10338 master.cc:561] Master@127.10.24.189:41829 shutting down...
I20250904 19:58:42.259001 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:42.259536 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:42.259883 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 99b0fcddcfce462594cf90b9dbc20f0b: stopping tablet replica
I20250904 19:58:42.278445 10338 master.cc:583] Master@127.10.24.189:41829 shutdown complete.
I20250904 19:58:42.292461 10338 master.cc:561] Master@127.10.24.188:39765 shutting down...
I20250904 19:58:42.308436 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:42.309293 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:42.309617 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P aa26145c3ed0401c8b1a6151c56f1c16: stopping tablet replica
I20250904 19:58:42.328913 10338 master.cc:583] Master@127.10.24.188:39765 shutdown complete.
[ OK ] RemoteKsckTest.TestChecksumSnapshotCurrentTimestamp (7263 ms)
[ RUN ] RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM
I20250904 19:58:42.363408 10338 test_util.cc:276] Using random seed: 636144637
I20250904 19:58:42.364343 10338 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.24.190:39577,127.10.24.189:40799,127.10.24.188:41125
I20250904 19:58:42.365607 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:42.370787 15246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:42.370918 15245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:42.371896 15248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:42.372892 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:42.373855 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:42.374034 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:42.374172 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015922374156 us; error 0 us; skew 500 ppm
I20250904 19:58:42.374662 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:42.376855 10338 webserver.cc:480] Webserver started at http://127.10.24.190:38561/ using document root <none> and password file <none>
I20250904 19:58:42.377324 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:42.377483 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:42.377722 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:42.378769 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/master-0-root/instance:
uuid: "af299e90d7cc4c9180cead65dedf1298"
format_stamp: "Formatted at 2025-09-04 19:58:42 on dist-test-slave-f3xj"
I20250904 19:58:42.382895 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20250904 19:58:42.386006 15253 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:42.386646 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250904 19:58:42.386900 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/master-0-root
uuid: "af299e90d7cc4c9180cead65dedf1298"
format_stamp: "Formatted at 2025-09-04 19:58:42 on dist-test-slave-f3xj"
I20250904 19:58:42.387241 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:42.411537 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:42.412811 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:42.447592 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.190:39577
I20250904 19:58:42.447703 15304 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.190:39577 every 8 connection(s)
I20250904 19:58:42.451134 15305 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:42.451303 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:42.456202 15307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:42.457197 15308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:42.459630 15310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:42.459832 15305 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:42.462606 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:42.463680 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:42.463858 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:42.464023 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015922464007 us; error 0 us; skew 500 ppm
I20250904 19:58:42.464661 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:42.467116 10338 webserver.cc:480] Webserver started at http://127.10.24.189:34999/ using document root <none> and password file <none>
I20250904 19:58:42.467700 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:42.467921 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:42.468227 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:42.469584 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/master-1-root/instance:
uuid: "65dd321cbb564039bb2b879cf12f82cf"
format_stamp: "Formatted at 2025-09-04 19:58:42 on dist-test-slave-f3xj"
I20250904 19:58:42.472491 15305 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:42.474663 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
W20250904 19:58:42.477315 15305 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.189:40799: Network error: Client connection negotiation failed: client connection to 127.10.24.189:40799: connect: Connection refused (error 111)
I20250904 19:58:42.478719 15318 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:42.479418 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250904 19:58:42.479699 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/master-1-root
uuid: "65dd321cbb564039bb2b879cf12f82cf"
format_stamp: "Formatted at 2025-09-04 19:58:42 on dist-test-slave-f3xj"
I20250904 19:58:42.479945 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:42.493924 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:42.494798 15305 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } attempt: 1
I20250904 19:58:42.494993 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
W20250904 19:58:42.498828 15305 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.189:40799: Network error: Client connection negotiation failed: client connection to 127.10.24.189:40799: connect: Connection refused (error 111)
I20250904 19:58:42.531642 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.189:40799
I20250904 19:58:42.531745 15369 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.189:40799 every 8 connection(s)
I20250904 19:58:42.535372 15370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:42.535239 15305 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } attempt: 2
I20250904 19:58:42.535480 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:42.542706 15374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:42.544317 15370 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:58:42.545248 15373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:42.549815 10338 server_base.cc:1047] running on GCE node
W20250904 19:58:42.551438 15376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:42.551963 15305 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:42.552418 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:42.552719 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:42.552920 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015922552902 us; error 0 us; skew 500 ppm
I20250904 19:58:42.553674 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
W20250904 19:58:42.557034 15305 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:41125: Network error: Client connection negotiation failed: client connection to 127.10.24.188:41125: connect: Connection refused (error 111)
I20250904 19:58:42.557055 10338 webserver.cc:480] Webserver started at http://127.10.24.188:39087/ using document root <none> and password file <none>
I20250904 19:58:42.557883 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:42.558185 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:42.558450 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:42.559325 15370 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:42.559495 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/master-2-root/instance:
uuid: "66dff7d2052c487da0691f348efce8fb"
format_stamp: "Formatted at 2025-09-04 19:58:42 on dist-test-slave-f3xj"
I20250904 19:58:42.564872 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20250904 19:58:42.568648 15370 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:42.568970 15382 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:42.569813 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20250904 19:58:42.570147 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/master-2-root
uuid: "66dff7d2052c487da0691f348efce8fb"
format_stamp: "Formatted at 2025-09-04 19:58:42 on dist-test-slave-f3xj"
I20250904 19:58:42.570472 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20250904 19:58:42.573305 15370 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:41125: Network error: Client connection negotiation failed: client connection to 127.10.24.188:41125: connect: Connection refused (error 111)
I20250904 19:58:42.589558 15305 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } attempt: 1
I20250904 19:58:42.590793 15370 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } attempt: 1
W20250904 19:58:42.593721 15305 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:41125: Network error: Client connection negotiation failed: client connection to 127.10.24.188:41125: connect: Connection refused (error 111)
I20250904 19:58:42.593935 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
W20250904 19:58:42.594681 15370 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:41125: Network error: Client connection negotiation failed: client connection to 127.10.24.188:41125: connect: Connection refused (error 111)
I20250904 19:58:42.595140 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:42.631583 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.188:41125
I20250904 19:58:42.631680 15434 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.188:41125 every 8 connection(s)
I20250904 19:58:42.633091 15370 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } attempt: 2
I20250904 19:58:42.635831 15435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:42.636204 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 19:58:42.642418 15435 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:42.652418 15370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf: Bootstrap starting.
I20250904 19:58:42.653062 15435 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:42.658231 15305 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } attempt: 2
I20250904 19:58:42.658562 15370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:42.662858 15435 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:42.664063 15370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf: No bootstrap required, opened a new log
I20250904 19:58:42.666994 15370 raft_consensus.cc:357] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af299e90d7cc4c9180cead65dedf1298" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } } peers { permanent_uuid: "65dd321cbb564039bb2b879cf12f82cf" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } } peers { permanent_uuid: "66dff7d2052c487da0691f348efce8fb" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } }
I20250904 19:58:42.667656 15370 raft_consensus.cc:383] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:42.667948 15370 raft_consensus.cc:738] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65dd321cbb564039bb2b879cf12f82cf, State: Initialized, Role: FOLLOWER
I20250904 19:58:42.668673 15370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af299e90d7cc4c9180cead65dedf1298" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } } peers { permanent_uuid: "65dd321cbb564039bb2b879cf12f82cf" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } } peers { permanent_uuid: "66dff7d2052c487da0691f348efce8fb" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } }
I20250904 19:58:42.670769 15441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af299e90d7cc4c9180cead65dedf1298" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } } peers { permanent_uuid: "65dd321cbb564039bb2b879cf12f82cf" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } } peers { permanent_uuid: "66dff7d2052c487da0691f348efce8fb" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } } }
I20250904 19:58:42.671618 15441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:42.672458 15370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:42.679071 15305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298: Bootstrap starting.
I20250904 19:58:42.681035 15435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb: Bootstrap starting.
I20250904 19:58:42.685305 15305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:42.687674 15435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:42.690019 15305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298: No bootstrap required, opened a new log
W20250904 19:58:42.691730 15452 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:58:42.691984 15452 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250904 19:58:42.692852 15435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb: No bootstrap required, opened a new log
I20250904 19:58:42.692487 15305 raft_consensus.cc:357] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af299e90d7cc4c9180cead65dedf1298" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } } peers { permanent_uuid: "65dd321cbb564039bb2b879cf12f82cf" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } } peers { permanent_uuid: "66dff7d2052c487da0691f348efce8fb" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } }
I20250904 19:58:42.693213 15305 raft_consensus.cc:383] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:42.693506 15305 raft_consensus.cc:738] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af299e90d7cc4c9180cead65dedf1298, State: Initialized, Role: FOLLOWER
I20250904 19:58:42.694200 15305 consensus_queue.cc:260] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af299e90d7cc4c9180cead65dedf1298" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } } peers { permanent_uuid: "65dd321cbb564039bb2b879cf12f82cf" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } } peers { permanent_uuid: "66dff7d2052c487da0691f348efce8fb" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } }
I20250904 19:58:42.695592 15435 raft_consensus.cc:357] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af299e90d7cc4c9180cead65dedf1298" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } } peers { permanent_uuid: "65dd321cbb564039bb2b879cf12f82cf" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } } peers { permanent_uuid: "66dff7d2052c487da0691f348efce8fb" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } }
I20250904 19:58:42.696089 15455 sys_catalog.cc:455] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af299e90d7cc4c9180cead65dedf1298" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } } peers { permanent_uuid: "65dd321cbb564039bb2b879cf12f82cf" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } } peers { permanent_uuid: "66dff7d2052c487da0691f348efce8fb" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } } }
I20250904 19:58:42.696344 15435 raft_consensus.cc:383] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:42.696668 15455 sys_catalog.cc:458] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:42.696746 15435 raft_consensus.cc:738] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66dff7d2052c487da0691f348efce8fb, State: Initialized, Role: FOLLOWER
I20250904 19:58:42.697546 15305 sys_catalog.cc:564] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:42.697448 15435 consensus_queue.cc:260] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af299e90d7cc4c9180cead65dedf1298" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } } peers { permanent_uuid: "65dd321cbb564039bb2b879cf12f82cf" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } } peers { permanent_uuid: "66dff7d2052c487da0691f348efce8fb" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } }
I20250904 19:58:42.699270 15456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af299e90d7cc4c9180cead65dedf1298" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } } peers { permanent_uuid: "65dd321cbb564039bb2b879cf12f82cf" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } } peers { permanent_uuid: "66dff7d2052c487da0691f348efce8fb" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } } }
I20250904 19:58:42.700064 15456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:42.700775 15435 sys_catalog.cc:564] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:42.709860 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20250904 19:58:42.710191 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
W20250904 19:58:42.713181 15473 catalog_manager.cc:1560] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:58:42.713467 15473 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250904 19:58:42.714711 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:42.716282 15478 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:58:42.716547 15478 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20250904 19:58:42.720911 15479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:42.721474 15480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:42.722743 15482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:42.723551 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:42.724373 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:42.724565 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:42.724712 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015922724696 us; error 0 us; skew 500 ppm
I20250904 19:58:42.725190 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:42.727315 10338 webserver.cc:480] Webserver started at http://127.10.24.129:33503/ using document root <none> and password file <none>
I20250904 19:58:42.727783 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:42.727949 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:42.728171 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:42.729121 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/ts-0-root/instance:
uuid: "7f14b896f4e94cd58c98621ff06f0242"
format_stamp: "Formatted at 2025-09-04 19:58:42 on dist-test-slave-f3xj"
I20250904 19:58:42.733108 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250904 19:58:42.736060 15487 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:42.736704 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250904 19:58:42.736967 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/ts-0-root
uuid: "7f14b896f4e94cd58c98621ff06f0242"
format_stamp: "Formatted at 2025-09-04 19:58:42 on dist-test-slave-f3xj"
I20250904 19:58:42.737228 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:42.754089 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:42.755242 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:42.756662 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:42.758774 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:42.758961 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:42.759173 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:42.759326 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:42.795992 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.129:39205
I20250904 19:58:42.796103 15549 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.129:39205 every 8 connection(s)
I20250904 19:58:42.804728 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:42.816792 15550 heartbeater.cc:344] Connected to a master server at 127.10.24.188:41125
I20250904 19:58:42.817219 15550 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:42.818082 15550 heartbeater.cc:507] Master 127.10.24.188:41125 requested a full tablet report, sending...
I20250904 19:58:42.821305 15400 ts_manager.cc:194] Registered new tserver with Master: 7f14b896f4e94cd58c98621ff06f0242 (127.10.24.129:39205)
W20250904 19:58:42.824563 15557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:42.824661 15551 heartbeater.cc:344] Connected to a master server at 127.10.24.189:40799
I20250904 19:58:42.825127 15551 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:42.825940 15551 heartbeater.cc:507] Master 127.10.24.189:40799 requested a full tablet report, sending...
W20250904 19:58:42.826102 15558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:42.827337 15552 heartbeater.cc:344] Connected to a master server at 127.10.24.190:39577
I20250904 19:58:42.827756 15552 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:42.828554 15552 heartbeater.cc:507] Master 127.10.24.190:39577 requested a full tablet report, sending...
I20250904 19:58:42.828763 15335 ts_manager.cc:194] Registered new tserver with Master: 7f14b896f4e94cd58c98621ff06f0242 (127.10.24.129:39205)
I20250904 19:58:42.830967 15270 ts_manager.cc:194] Registered new tserver with Master: 7f14b896f4e94cd58c98621ff06f0242 (127.10.24.129:39205)
W20250904 19:58:42.833684 15560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:42.833890 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:42.834779 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:42.834980 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:42.835140 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015922835127 us; error 0 us; skew 500 ppm
I20250904 19:58:42.835695 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:42.837949 10338 webserver.cc:480] Webserver started at http://127.10.24.130:37191/ using document root <none> and password file <none>
I20250904 19:58:42.838404 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:42.838577 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:42.838852 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:42.839915 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/ts-1-root/instance:
uuid: "e3a0aa52a3cc41489dd8d20149971578"
format_stamp: "Formatted at 2025-09-04 19:58:42 on dist-test-slave-f3xj"
I20250904 19:58:42.845417 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20250904 19:58:42.848912 15565 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:42.849586 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250904 19:58:42.849861 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/ts-1-root
uuid: "e3a0aa52a3cc41489dd8d20149971578"
format_stamp: "Formatted at 2025-09-04 19:58:42 on dist-test-slave-f3xj"
I20250904 19:58:42.850104 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:42.862255 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:42.863319 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:42.864707 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:42.866773 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:42.867027 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:42.867301 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:42.867522 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:42.913064 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.130:38253
I20250904 19:58:42.913651 15627 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.130:38253 every 8 connection(s)
I20250904 19:58:42.919874 15455 raft_consensus.cc:491] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:42.920456 15455 raft_consensus.cc:513] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af299e90d7cc4c9180cead65dedf1298" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } } peers { permanent_uuid: "65dd321cbb564039bb2b879cf12f82cf" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } } peers { permanent_uuid: "66dff7d2052c487da0691f348efce8fb" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } }
I20250904 19:58:42.922569 15455 leader_election.cc:290] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 65dd321cbb564039bb2b879cf12f82cf (127.10.24.189:40799), 66dff7d2052c487da0691f348efce8fb (127.10.24.188:41125)
I20250904 19:58:42.927553 15410 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af299e90d7cc4c9180cead65dedf1298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66dff7d2052c487da0691f348efce8fb" is_pre_election: true
I20250904 19:58:42.928364 15410 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af299e90d7cc4c9180cead65dedf1298 in term 0.
I20250904 19:58:42.928984 15345 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af299e90d7cc4c9180cead65dedf1298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dd321cbb564039bb2b879cf12f82cf" is_pre_election: true
I20250904 19:58:42.929620 15257 leader_election.cc:304] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66dff7d2052c487da0691f348efce8fb, af299e90d7cc4c9180cead65dedf1298; no voters:
I20250904 19:58:42.929709 15345 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af299e90d7cc4c9180cead65dedf1298 in term 0.
I20250904 19:58:42.932381 15455 raft_consensus.cc:2802] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:42.932827 15455 raft_consensus.cc:491] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:42.933424 15455 raft_consensus.cc:3058] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:42.935191 15628 heartbeater.cc:344] Connected to a master server at 127.10.24.188:41125
I20250904 19:58:42.935612 15628 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:42.936455 15628 heartbeater.cc:507] Master 127.10.24.188:41125 requested a full tablet report, sending...
I20250904 19:58:42.939153 15400 ts_manager.cc:194] Registered new tserver with Master: e3a0aa52a3cc41489dd8d20149971578 (127.10.24.130:38253)
I20250904 19:58:42.939294 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:42.949234 15629 heartbeater.cc:344] Connected to a master server at 127.10.24.189:40799
I20250904 19:58:42.949700 15629 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:42.950628 15629 heartbeater.cc:507] Master 127.10.24.189:40799 requested a full tablet report, sending...
I20250904 19:58:42.948338 15455 raft_consensus.cc:513] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af299e90d7cc4c9180cead65dedf1298" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } } peers { permanent_uuid: "65dd321cbb564039bb2b879cf12f82cf" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } } peers { permanent_uuid: "66dff7d2052c487da0691f348efce8fb" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } }
I20250904 19:58:42.953481 15335 ts_manager.cc:194] Registered new tserver with Master: e3a0aa52a3cc41489dd8d20149971578 (127.10.24.130:38253)
I20250904 19:58:42.956923 15345 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af299e90d7cc4c9180cead65dedf1298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65dd321cbb564039bb2b879cf12f82cf"
I20250904 19:58:42.957562 15345 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:42.957706 15455 leader_election.cc:290] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [CANDIDATE]: Term 1 election: Requested vote from peers 65dd321cbb564039bb2b879cf12f82cf (127.10.24.189:40799), 66dff7d2052c487da0691f348efce8fb (127.10.24.188:41125)
I20250904 19:58:42.958696 15410 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af299e90d7cc4c9180cead65dedf1298" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66dff7d2052c487da0691f348efce8fb"
I20250904 19:58:42.960498 15630 heartbeater.cc:344] Connected to a master server at 127.10.24.190:39577
I20250904 19:58:42.960865 15410 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:42.960954 15630 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:42.968329 15630 heartbeater.cc:507] Master 127.10.24.190:39577 requested a full tablet report, sending...
W20250904 19:58:42.960211 15634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:42.968326 15410 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af299e90d7cc4c9180cead65dedf1298 in term 1.
W20250904 19:58:42.970110 15635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:42.971107 15638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:42.971421 15270 ts_manager.cc:194] Registered new tserver with Master: e3a0aa52a3cc41489dd8d20149971578 (127.10.24.130:38253)
I20250904 19:58:42.972651 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:42.972960 15257 leader_election.cc:304] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66dff7d2052c487da0691f348efce8fb, af299e90d7cc4c9180cead65dedf1298; no voters:
I20250904 19:58:42.973927 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:42.974280 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:42.974220 15345 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af299e90d7cc4c9180cead65dedf1298 in term 1.
I20250904 19:58:42.974612 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015922974593 us; error 0 us; skew 500 ppm
I20250904 19:58:42.975546 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:42.975787 15455 raft_consensus.cc:2802] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:42.977267 15455 raft_consensus.cc:695] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [term 1 LEADER]: Becoming Leader. State: Replica: af299e90d7cc4c9180cead65dedf1298, State: Running, Role: LEADER
I20250904 19:58:42.978180 15455 consensus_queue.cc:237] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af299e90d7cc4c9180cead65dedf1298" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } } peers { permanent_uuid: "65dd321cbb564039bb2b879cf12f82cf" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } } peers { permanent_uuid: "66dff7d2052c487da0691f348efce8fb" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } }
I20250904 19:58:42.980370 10338 webserver.cc:480] Webserver started at http://127.10.24.131:39445/ using document root <none> and password file <none>
I20250904 19:58:42.981060 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:42.981302 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:42.981581 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:42.983083 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/ts-2-root/instance:
uuid: "3c867a82454b4215a6176c32909df17d"
format_stamp: "Formatted at 2025-09-04 19:58:42 on dist-test-slave-f3xj"
I20250904 19:58:42.986369 15639 sys_catalog.cc:455] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af299e90d7cc4c9180cead65dedf1298. Latest consensus state: current_term: 1 leader_uuid: "af299e90d7cc4c9180cead65dedf1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af299e90d7cc4c9180cead65dedf1298" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } } peers { permanent_uuid: "65dd321cbb564039bb2b879cf12f82cf" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } } peers { permanent_uuid: "66dff7d2052c487da0691f348efce8fb" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } } }
I20250904 19:58:42.987098 15639 sys_catalog.cc:458] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:42.992631 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.009s user 0.004s sys 0.003s
I20250904 19:58:42.996464 15643 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 19:58:42.997814 15645 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:43.000303 10338 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20250904 19:58:43.000687 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/ts-2-root
uuid: "3c867a82454b4215a6176c32909df17d"
format_stamp: "Formatted at 2025-09-04 19:58:42 on dist-test-slave-f3xj"
I20250904 19:58:43.001132 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM.1757015869512910-10338-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:43.004861 15643 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 19:58:43.017897 15345 raft_consensus.cc:1273] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf [term 1 FOLLOWER]: Refusing update from remote peer af299e90d7cc4c9180cead65dedf1298: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:43.019160 15410 raft_consensus.cc:1273] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb [term 1 FOLLOWER]: Refusing update from remote peer af299e90d7cc4c9180cead65dedf1298: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:43.019212 15455 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65dd321cbb564039bb2b879cf12f82cf" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250904 19:58:43.023844 15455 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [LEADER]: Connected to new peer: Peer: permanent_uuid: "66dff7d2052c487da0691f348efce8fb" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:43.030395 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:43.032150 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:43.032351 15456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader af299e90d7cc4c9180cead65dedf1298. Latest consensus state: current_term: 1 leader_uuid: "af299e90d7cc4c9180cead65dedf1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af299e90d7cc4c9180cead65dedf1298" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } } peers { permanent_uuid: "65dd321cbb564039bb2b879cf12f82cf" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } } peers { permanent_uuid: "66dff7d2052c487da0691f348efce8fb" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } } }
I20250904 19:58:43.032989 15456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:43.034045 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:43.035211 15441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader af299e90d7cc4c9180cead65dedf1298. Latest consensus state: current_term: 1 leader_uuid: "af299e90d7cc4c9180cead65dedf1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af299e90d7cc4c9180cead65dedf1298" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } } peers { permanent_uuid: "65dd321cbb564039bb2b879cf12f82cf" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } } peers { permanent_uuid: "66dff7d2052c487da0691f348efce8fb" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } } }
I20250904 19:58:43.035984 15441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:43.044090 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:43.044490 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20250904 19:58:43.044817 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:43.045038 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:43.049671 15456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af299e90d7cc4c9180cead65dedf1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af299e90d7cc4c9180cead65dedf1298" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } } peers { permanent_uuid: "65dd321cbb564039bb2b879cf12f82cf" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } } peers { permanent_uuid: "66dff7d2052c487da0691f348efce8fb" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } } }
I20250904 19:58:43.050451 15456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:43.056654 15441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af299e90d7cc4c9180cead65dedf1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af299e90d7cc4c9180cead65dedf1298" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } } peers { permanent_uuid: "65dd321cbb564039bb2b879cf12f82cf" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } } peers { permanent_uuid: "66dff7d2052c487da0691f348efce8fb" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } } }
I20250904 19:58:43.057395 15441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:43.062129 15643 catalog_manager.cc:1349] Generated new cluster ID: 37c445190445403cb48ca3d7be9c307d
I20250904 19:58:43.062463 15643 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 19:58:43.065174 15639 sys_catalog.cc:455] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af299e90d7cc4c9180cead65dedf1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af299e90d7cc4c9180cead65dedf1298" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } } peers { permanent_uuid: "65dd321cbb564039bb2b879cf12f82cf" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } } peers { permanent_uuid: "66dff7d2052c487da0691f348efce8fb" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } } }
I20250904 19:58:43.066228 15455 sys_catalog.cc:455] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af299e90d7cc4c9180cead65dedf1298" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af299e90d7cc4c9180cead65dedf1298" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 39577 } } peers { permanent_uuid: "65dd321cbb564039bb2b879cf12f82cf" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 40799 } } peers { permanent_uuid: "66dff7d2052c487da0691f348efce8fb" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 41125 } } }
I20250904 19:58:43.067025 15455 sys_catalog.cc:458] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:43.067497 15639 sys_catalog.cc:458] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:43.102641 15643 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 19:58:43.104483 15643 catalog_manager.cc:1506] Loading token signing keys...
I20250904 19:58:43.136278 15643 catalog_manager.cc:5955] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298: Generated new TSK 0
I20250904 19:58:43.136966 15643 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 19:58:43.154362 15269 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56854
I20250904 19:58:43.159689 15269 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56864
I20250904 19:58:43.172292 15552 heartbeater.cc:499] Master 127.10.24.190:39577 was elected leader, sending a full tablet report...
I20250904 19:58:43.172379 15630 heartbeater.cc:499] Master 127.10.24.190:39577 was elected leader, sending a full tablet report...
I20250904 19:58:43.192112 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.131:33469
I20250904 19:58:43.192158 15716 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.131:33469 every 8 connection(s)
I20250904 19:58:43.235844 15717 heartbeater.cc:344] Connected to a master server at 127.10.24.188:41125
I20250904 19:58:43.236330 15717 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:43.237250 15717 heartbeater.cc:507] Master 127.10.24.188:41125 requested a full tablet report, sending...
I20250904 19:58:43.241058 15399 ts_manager.cc:194] Registered new tserver with Master: 3c867a82454b4215a6176c32909df17d (127.10.24.131:33469)
I20250904 19:58:43.242514 15720 heartbeater.cc:344] Connected to a master server at 127.10.24.190:39577
I20250904 19:58:43.243408 15718 heartbeater.cc:344] Connected to a master server at 127.10.24.189:40799
I20250904 19:58:43.243593 15720 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:43.243826 15718 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:43.244442 15720 heartbeater.cc:507] Master 127.10.24.190:39577 requested a full tablet report, sending...
I20250904 19:58:43.244606 15718 heartbeater.cc:507] Master 127.10.24.189:40799 requested a full tablet report, sending...
I20250904 19:58:43.246857 15269 ts_manager.cc:194] Registered new tserver with Master: 3c867a82454b4215a6176c32909df17d (127.10.24.131:33469)
I20250904 19:58:43.247337 15334 ts_manager.cc:194] Registered new tserver with Master: 3c867a82454b4215a6176c32909df17d (127.10.24.131:33469)
I20250904 19:58:43.247772 10338 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.023760699s
I20250904 19:58:43.249102 15269 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56878
I20250904 19:58:43.261930 15720 heartbeater.cc:499] Master 127.10.24.190:39577 was elected leader, sending a full tablet report...
I20250904 19:58:43.301771 15270 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56880:
name: "ksck-test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\n\000\000\000\004\001\024\000\000\000""\004\001\n\000\000\000\004\001\024\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 19:58:43.304836 15270 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table ksck-test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 19:58:43.408466 15682 tablet_service.cc:1468] Processing CreateTablet for tablet 63602651692d46ac96799a909542ec00 (DEFAULT_TABLE table=ksck-test-table [id=fac352434b034a8188697e6dc2460a50]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:43.409907 15682 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63602651692d46ac96799a909542ec00. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:43.410152 15681 tablet_service.cc:1468] Processing CreateTablet for tablet 0591d48fddb94304bbcf561723dd0e95 (DEFAULT_TABLE table=ksck-test-table [id=fac352434b034a8188697e6dc2460a50]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:43.411504 15681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0591d48fddb94304bbcf561723dd0e95. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:43.415426 15593 tablet_service.cc:1468] Processing CreateTablet for tablet 63602651692d46ac96799a909542ec00 (DEFAULT_TABLE table=ksck-test-table [id=fac352434b034a8188697e6dc2460a50]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:43.416812 15593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63602651692d46ac96799a909542ec00. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:43.419106 15592 tablet_service.cc:1468] Processing CreateTablet for tablet 0591d48fddb94304bbcf561723dd0e95 (DEFAULT_TABLE table=ksck-test-table [id=fac352434b034a8188697e6dc2460a50]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:43.420466 15592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0591d48fddb94304bbcf561723dd0e95. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:43.423600 15591 tablet_service.cc:1468] Processing CreateTablet for tablet 6b808d2abc454da28f9a53ce9d46b3e5 (DEFAULT_TABLE table=ksck-test-table [id=fac352434b034a8188697e6dc2460a50]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:43.435940 15591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b808d2abc454da28f9a53ce9d46b3e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:43.435992 15515 tablet_service.cc:1468] Processing CreateTablet for tablet 63602651692d46ac96799a909542ec00 (DEFAULT_TABLE table=ksck-test-table [id=fac352434b034a8188697e6dc2460a50]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:43.436761 15513 tablet_service.cc:1468] Processing CreateTablet for tablet 6b808d2abc454da28f9a53ce9d46b3e5 (DEFAULT_TABLE table=ksck-test-table [id=fac352434b034a8188697e6dc2460a50]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:43.438316 15514 tablet_service.cc:1468] Processing CreateTablet for tablet 0591d48fddb94304bbcf561723dd0e95 (DEFAULT_TABLE table=ksck-test-table [id=fac352434b034a8188697e6dc2460a50]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:43.437265 15515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63602651692d46ac96799a909542ec00. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:43.453730 15514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0591d48fddb94304bbcf561723dd0e95. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:43.454622 15513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b808d2abc454da28f9a53ce9d46b3e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:43.481104 15680 tablet_service.cc:1468] Processing CreateTablet for tablet 6b808d2abc454da28f9a53ce9d46b3e5 (DEFAULT_TABLE table=ksck-test-table [id=fac352434b034a8188697e6dc2460a50]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:43.482676 15680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b808d2abc454da28f9a53ce9d46b3e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:43.511793 15738 tablet_bootstrap.cc:492] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d: Bootstrap starting.
I20250904 19:58:43.517861 15738 tablet_bootstrap.cc:654] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:43.539613 15739 tablet_bootstrap.cc:492] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578: Bootstrap starting.
I20250904 19:58:43.590652 15741 tablet_bootstrap.cc:492] T 63602651692d46ac96799a909542ec00 P 7f14b896f4e94cd58c98621ff06f0242: Bootstrap starting.
I20250904 19:58:43.593864 15739 tablet_bootstrap.cc:654] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:43.597661 15738 tablet_bootstrap.cc:492] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d: No bootstrap required, opened a new log
I20250904 19:58:43.598136 15741 tablet_bootstrap.cc:654] T 63602651692d46ac96799a909542ec00 P 7f14b896f4e94cd58c98621ff06f0242: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:43.603154 15738 ts_tablet_manager.cc:1397] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d: Time spent bootstrapping tablet: real 0.092s user 0.035s sys 0.043s
I20250904 19:58:43.606105 15738 raft_consensus.cc:357] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } } peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } }
I20250904 19:58:43.606742 15738 raft_consensus.cc:383] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:43.606943 15738 raft_consensus.cc:738] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c867a82454b4215a6176c32909df17d, State: Initialized, Role: FOLLOWER
I20250904 19:58:43.607481 15738 consensus_queue.cc:260] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } } peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } }
I20250904 19:58:43.610740 15739 tablet_bootstrap.cc:492] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578: No bootstrap required, opened a new log
I20250904 19:58:43.611233 15739 ts_tablet_manager.cc:1397] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578: Time spent bootstrapping tablet: real 0.088s user 0.000s sys 0.010s
I20250904 19:58:43.619881 15739 raft_consensus.cc:357] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } } peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } }
I20250904 19:58:43.620602 15739 raft_consensus.cc:383] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:43.620879 15739 raft_consensus.cc:738] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3a0aa52a3cc41489dd8d20149971578, State: Initialized, Role: FOLLOWER
I20250904 19:58:43.621634 15739 consensus_queue.cc:260] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } } peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } }
I20250904 19:58:43.641438 15739 ts_tablet_manager.cc:1428] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578: Time spent starting tablet: real 0.024s user 0.006s sys 0.006s
I20250904 19:58:43.696511 15738 ts_tablet_manager.cc:1428] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d: Time spent starting tablet: real 0.093s user 0.013s sys 0.023s
I20250904 19:58:43.696986 15452 catalog_manager.cc:1261] Loaded cluster ID: 37c445190445403cb48ca3d7be9c307d
I20250904 19:58:43.697288 15452 catalog_manager.cc:1554] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf: loading cluster ID for follower catalog manager: success
I20250904 19:58:43.697680 15738 tablet_bootstrap.cc:492] T 6b808d2abc454da28f9a53ce9d46b3e5 P 3c867a82454b4215a6176c32909df17d: Bootstrap starting.
I20250904 19:58:43.702052 15739 tablet_bootstrap.cc:492] T 63602651692d46ac96799a909542ec00 P e3a0aa52a3cc41489dd8d20149971578: Bootstrap starting.
I20250904 19:58:43.704267 15452 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf: acquiring CA information for follower catalog manager: success
I20250904 19:58:43.709586 15741 tablet_bootstrap.cc:492] T 63602651692d46ac96799a909542ec00 P 7f14b896f4e94cd58c98621ff06f0242: No bootstrap required, opened a new log
I20250904 19:58:43.710141 15741 ts_tablet_manager.cc:1397] T 63602651692d46ac96799a909542ec00 P 7f14b896f4e94cd58c98621ff06f0242: Time spent bootstrapping tablet: real 0.120s user 0.026s sys 0.049s
I20250904 19:58:43.712805 15738 tablet_bootstrap.cc:654] T 6b808d2abc454da28f9a53ce9d46b3e5 P 3c867a82454b4215a6176c32909df17d: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:43.713002 15741 raft_consensus.cc:357] T 63602651692d46ac96799a909542ec00 P 7f14b896f4e94cd58c98621ff06f0242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } } peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } }
I20250904 19:58:43.713714 15741 raft_consensus.cc:383] T 63602651692d46ac96799a909542ec00 P 7f14b896f4e94cd58c98621ff06f0242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:43.714006 15741 raft_consensus.cc:738] T 63602651692d46ac96799a909542ec00 P 7f14b896f4e94cd58c98621ff06f0242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f14b896f4e94cd58c98621ff06f0242, State: Initialized, Role: FOLLOWER
I20250904 19:58:43.714687 15741 consensus_queue.cc:260] T 63602651692d46ac96799a909542ec00 P 7f14b896f4e94cd58c98621ff06f0242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } } peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } }
I20250904 19:58:43.722877 15478 catalog_manager.cc:1261] Loaded cluster ID: 37c445190445403cb48ca3d7be9c307d
I20250904 19:58:43.723250 15478 catalog_manager.cc:1554] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb: loading cluster ID for follower catalog manager: success
I20250904 19:58:43.726946 15739 tablet_bootstrap.cc:654] T 63602651692d46ac96799a909542ec00 P e3a0aa52a3cc41489dd8d20149971578: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:43.729715 15738 tablet_bootstrap.cc:492] T 6b808d2abc454da28f9a53ce9d46b3e5 P 3c867a82454b4215a6176c32909df17d: No bootstrap required, opened a new log
I20250904 19:58:43.730198 15738 ts_tablet_manager.cc:1397] T 6b808d2abc454da28f9a53ce9d46b3e5 P 3c867a82454b4215a6176c32909df17d: Time spent bootstrapping tablet: real 0.033s user 0.009s sys 0.008s
I20250904 19:58:43.730491 15452 catalog_manager.cc:1604] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:58:43.735255 15478 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb: acquiring CA information for follower catalog manager: success
I20250904 19:58:43.737030 15738 raft_consensus.cc:357] T 6b808d2abc454da28f9a53ce9d46b3e5 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } } peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } }
I20250904 19:58:43.737753 15738 raft_consensus.cc:383] T 6b808d2abc454da28f9a53ce9d46b3e5 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:43.738523 15738 raft_consensus.cc:738] T 6b808d2abc454da28f9a53ce9d46b3e5 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c867a82454b4215a6176c32909df17d, State: Initialized, Role: FOLLOWER
I20250904 19:58:43.739394 15738 consensus_queue.cc:260] T 6b808d2abc454da28f9a53ce9d46b3e5 P 3c867a82454b4215a6176c32909df17d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } } peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } }
I20250904 19:58:43.741719 15738 ts_tablet_manager.cc:1428] T 6b808d2abc454da28f9a53ce9d46b3e5 P 3c867a82454b4215a6176c32909df17d: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20250904 19:58:43.742807 15738 tablet_bootstrap.cc:492] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d: Bootstrap starting.
I20250904 19:58:43.748919 15478 catalog_manager.cc:1604] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:58:43.752341 15741 ts_tablet_manager.cc:1428] T 63602651692d46ac96799a909542ec00 P 7f14b896f4e94cd58c98621ff06f0242: Time spent starting tablet: real 0.042s user 0.008s sys 0.004s
I20250904 19:58:43.753393 15741 tablet_bootstrap.cc:492] T 6b808d2abc454da28f9a53ce9d46b3e5 P 7f14b896f4e94cd58c98621ff06f0242: Bootstrap starting.
I20250904 19:58:43.756166 15738 tablet_bootstrap.cc:654] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:43.767792 15741 tablet_bootstrap.cc:654] T 6b808d2abc454da28f9a53ce9d46b3e5 P 7f14b896f4e94cd58c98621ff06f0242: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:43.771039 15745 raft_consensus.cc:491] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:43.771785 15738 tablet_bootstrap.cc:492] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d: No bootstrap required, opened a new log
I20250904 19:58:43.771546 15745 raft_consensus.cc:513] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } } peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } }
I20250904 19:58:43.779659 15745 leader_election.cc:290] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14b896f4e94cd58c98621ff06f0242 (127.10.24.129:39205), 3c867a82454b4215a6176c32909df17d (127.10.24.131:33469)
I20250904 19:58:43.786564 15739 tablet_bootstrap.cc:492] T 63602651692d46ac96799a909542ec00 P e3a0aa52a3cc41489dd8d20149971578: No bootstrap required, opened a new log
I20250904 19:58:43.787122 15739 ts_tablet_manager.cc:1397] T 63602651692d46ac96799a909542ec00 P e3a0aa52a3cc41489dd8d20149971578: Time spent bootstrapping tablet: real 0.085s user 0.016s sys 0.008s
I20250904 19:58:43.799883 15738 ts_tablet_manager.cc:1397] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d: Time spent bootstrapping tablet: real 0.057s user 0.014s sys 0.003s
I20250904 19:58:43.799754 15739 raft_consensus.cc:357] T 63602651692d46ac96799a909542ec00 P e3a0aa52a3cc41489dd8d20149971578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } } peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } }
I20250904 19:58:43.800530 15739 raft_consensus.cc:383] T 63602651692d46ac96799a909542ec00 P e3a0aa52a3cc41489dd8d20149971578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:43.800850 15739 raft_consensus.cc:738] T 63602651692d46ac96799a909542ec00 P e3a0aa52a3cc41489dd8d20149971578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3a0aa52a3cc41489dd8d20149971578, State: Initialized, Role: FOLLOWER
I20250904 19:58:43.801512 15739 consensus_queue.cc:260] T 63602651692d46ac96799a909542ec00 P e3a0aa52a3cc41489dd8d20149971578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } } peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } }
I20250904 19:58:43.803086 15738 raft_consensus.cc:357] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } } peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } }
I20250904 19:58:43.803781 15739 ts_tablet_manager.cc:1428] T 63602651692d46ac96799a909542ec00 P e3a0aa52a3cc41489dd8d20149971578: Time spent starting tablet: real 0.016s user 0.003s sys 0.006s
I20250904 19:58:43.803812 15738 raft_consensus.cc:383] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:43.809429 15738 raft_consensus.cc:738] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c867a82454b4215a6176c32909df17d, State: Initialized, Role: FOLLOWER
I20250904 19:58:43.810119 15738 consensus_queue.cc:260] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } } peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } }
I20250904 19:58:43.812309 15739 tablet_bootstrap.cc:492] T 0591d48fddb94304bbcf561723dd0e95 P e3a0aa52a3cc41489dd8d20149971578: Bootstrap starting.
I20250904 19:58:43.813272 15738 ts_tablet_manager.cc:1428] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20250904 19:58:43.849025 15743 raft_consensus.cc:491] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:43.849613 15743 raft_consensus.cc:513] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } } peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } }
I20250904 19:58:43.852205 15743 leader_election.cc:290] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f14b896f4e94cd58c98621ff06f0242 (127.10.24.129:39205), e3a0aa52a3cc41489dd8d20149971578 (127.10.24.130:38253)
I20250904 19:58:43.861315 15739 tablet_bootstrap.cc:654] T 0591d48fddb94304bbcf561723dd0e95 P e3a0aa52a3cc41489dd8d20149971578: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:43.875356 15739 tablet_bootstrap.cc:492] T 0591d48fddb94304bbcf561723dd0e95 P e3a0aa52a3cc41489dd8d20149971578: No bootstrap required, opened a new log
I20250904 19:58:43.875913 15739 ts_tablet_manager.cc:1397] T 0591d48fddb94304bbcf561723dd0e95 P e3a0aa52a3cc41489dd8d20149971578: Time spent bootstrapping tablet: real 0.064s user 0.007s sys 0.005s
I20250904 19:58:43.879217 15739 raft_consensus.cc:357] T 0591d48fddb94304bbcf561723dd0e95 P e3a0aa52a3cc41489dd8d20149971578 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } } peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } }
I20250904 19:58:43.880774 15525 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6b808d2abc454da28f9a53ce9d46b3e5" candidate_uuid: "e3a0aa52a3cc41489dd8d20149971578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f14b896f4e94cd58c98621ff06f0242" is_pre_election: true
W20250904 19:58:43.882863 15566 leader_election.cc:343] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14b896f4e94cd58c98621ff06f0242 (127.10.24.129:39205): Illegal state: must be running to vote when last-logged opid is not known
I20250904 19:58:43.879952 15739 raft_consensus.cc:383] T 0591d48fddb94304bbcf561723dd0e95 P e3a0aa52a3cc41489dd8d20149971578 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:43.884474 15739 raft_consensus.cc:738] T 0591d48fddb94304bbcf561723dd0e95 P e3a0aa52a3cc41489dd8d20149971578 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3a0aa52a3cc41489dd8d20149971578, State: Initialized, Role: FOLLOWER
I20250904 19:58:43.885216 15739 consensus_queue.cc:260] T 0591d48fddb94304bbcf561723dd0e95 P e3a0aa52a3cc41489dd8d20149971578 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } } peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } }
I20250904 19:58:43.887519 15739 ts_tablet_manager.cc:1428] T 0591d48fddb94304bbcf561723dd0e95 P e3a0aa52a3cc41489dd8d20149971578: Time spent starting tablet: real 0.011s user 0.004s sys 0.000s
I20250904 19:58:43.874153 15741 tablet_bootstrap.cc:492] T 6b808d2abc454da28f9a53ce9d46b3e5 P 7f14b896f4e94cd58c98621ff06f0242: No bootstrap required, opened a new log
I20250904 19:58:43.890268 15741 ts_tablet_manager.cc:1397] T 6b808d2abc454da28f9a53ce9d46b3e5 P 7f14b896f4e94cd58c98621ff06f0242: Time spent bootstrapping tablet: real 0.137s user 0.037s sys 0.053s
I20250904 19:58:43.893052 15741 raft_consensus.cc:357] T 6b808d2abc454da28f9a53ce9d46b3e5 P 7f14b896f4e94cd58c98621ff06f0242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } } peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } }
I20250904 19:58:43.893725 15741 raft_consensus.cc:383] T 6b808d2abc454da28f9a53ce9d46b3e5 P 7f14b896f4e94cd58c98621ff06f0242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:43.894004 15741 raft_consensus.cc:738] T 6b808d2abc454da28f9a53ce9d46b3e5 P 7f14b896f4e94cd58c98621ff06f0242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f14b896f4e94cd58c98621ff06f0242, State: Initialized, Role: FOLLOWER
I20250904 19:58:43.900696 15741 consensus_queue.cc:260] T 6b808d2abc454da28f9a53ce9d46b3e5 P 7f14b896f4e94cd58c98621ff06f0242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } } peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } }
I20250904 19:58:43.917881 15741 ts_tablet_manager.cc:1428] T 6b808d2abc454da28f9a53ce9d46b3e5 P 7f14b896f4e94cd58c98621ff06f0242: Time spent starting tablet: real 0.027s user 0.005s sys 0.001s
I20250904 19:58:43.918730 15692 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6b808d2abc454da28f9a53ce9d46b3e5" candidate_uuid: "e3a0aa52a3cc41489dd8d20149971578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c867a82454b4215a6176c32909df17d" is_pre_election: true
I20250904 19:58:43.919467 15692 raft_consensus.cc:2466] T 6b808d2abc454da28f9a53ce9d46b3e5 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3a0aa52a3cc41489dd8d20149971578 in term 0.
I20250904 19:58:43.920791 15568 leader_election.cc:304] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c867a82454b4215a6176c32909df17d, e3a0aa52a3cc41489dd8d20149971578; no voters: 7f14b896f4e94cd58c98621ff06f0242
I20250904 19:58:43.921825 15741 tablet_bootstrap.cc:492] T 0591d48fddb94304bbcf561723dd0e95 P 7f14b896f4e94cd58c98621ff06f0242: Bootstrap starting.
I20250904 19:58:43.922135 15525 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "0591d48fddb94304bbcf561723dd0e95" candidate_uuid: "3c867a82454b4215a6176c32909df17d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f14b896f4e94cd58c98621ff06f0242" is_pre_election: true
I20250904 19:58:43.923128 15745 raft_consensus.cc:2802] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:43.923521 15745 raft_consensus.cc:491] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:43.923899 15745 raft_consensus.cc:3058] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:43.932165 15745 raft_consensus.cc:513] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } } peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } }
I20250904 19:58:43.934844 15525 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6b808d2abc454da28f9a53ce9d46b3e5" candidate_uuid: "e3a0aa52a3cc41489dd8d20149971578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f14b896f4e94cd58c98621ff06f0242"
I20250904 19:58:43.935482 15525 raft_consensus.cc:3058] T 6b808d2abc454da28f9a53ce9d46b3e5 P 7f14b896f4e94cd58c98621ff06f0242 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:43.936399 15741 tablet_bootstrap.cc:654] T 0591d48fddb94304bbcf561723dd0e95 P 7f14b896f4e94cd58c98621ff06f0242: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:43.935014 15743 raft_consensus.cc:491] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:43.937342 15743 raft_consensus.cc:513] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } } peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } }
I20250904 19:58:43.939268 15743 leader_election.cc:290] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3a0aa52a3cc41489dd8d20149971578 (127.10.24.130:38253), 7f14b896f4e94cd58c98621ff06f0242 (127.10.24.129:39205)
I20250904 19:58:43.946139 15525 raft_consensus.cc:2466] T 6b808d2abc454da28f9a53ce9d46b3e5 P 7f14b896f4e94cd58c98621ff06f0242 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3a0aa52a3cc41489dd8d20149971578 in term 1.
I20250904 19:58:43.947355 15566 leader_election.cc:304] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f14b896f4e94cd58c98621ff06f0242, e3a0aa52a3cc41489dd8d20149971578; no voters:
I20250904 19:58:43.955092 15524 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "63602651692d46ac96799a909542ec00" candidate_uuid: "3c867a82454b4215a6176c32909df17d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f14b896f4e94cd58c98621ff06f0242" is_pre_election: true
I20250904 19:58:43.955852 15524 raft_consensus.cc:2466] T 63602651692d46ac96799a909542ec00 P 7f14b896f4e94cd58c98621ff06f0242 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c867a82454b4215a6176c32909df17d in term 0.
W20250904 19:58:43.955910 15646 leader_election.cc:343] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7f14b896f4e94cd58c98621ff06f0242 (127.10.24.129:39205): Illegal state: must be running to vote when last-logged opid is not known
I20250904 19:58:43.958014 15646 leader_election.cc:304] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c867a82454b4215a6176c32909df17d, 7f14b896f4e94cd58c98621ff06f0242; no voters:
I20250904 19:58:43.959169 15745 leader_election.cc:290] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [CANDIDATE]: Term 1 election: Requested vote from peers 7f14b896f4e94cd58c98621ff06f0242 (127.10.24.129:39205), 3c867a82454b4215a6176c32909df17d (127.10.24.131:33469)
I20250904 19:58:43.959478 15692 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6b808d2abc454da28f9a53ce9d46b3e5" candidate_uuid: "e3a0aa52a3cc41489dd8d20149971578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c867a82454b4215a6176c32909df17d"
I20250904 19:58:43.960103 15692 raft_consensus.cc:3058] T 6b808d2abc454da28f9a53ce9d46b3e5 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:43.960248 15745 raft_consensus.cc:2802] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:43.960728 15745 raft_consensus.cc:695] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [term 1 LEADER]: Becoming Leader. State: Replica: e3a0aa52a3cc41489dd8d20149971578, State: Running, Role: LEADER
I20250904 19:58:43.961531 15745 consensus_queue.cc:237] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } } peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } }
I20250904 19:58:43.963663 15743 raft_consensus.cc:2802] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:43.964056 15743 raft_consensus.cc:491] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:43.964383 15743 raft_consensus.cc:3058] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:43.971599 15743 raft_consensus.cc:513] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } } peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } }
I20250904 19:58:43.977770 15692 raft_consensus.cc:2466] T 6b808d2abc454da28f9a53ce9d46b3e5 P 3c867a82454b4215a6176c32909df17d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3a0aa52a3cc41489dd8d20149971578 in term 1.
I20250904 19:58:43.978295 15743 leader_election.cc:290] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d [CANDIDATE]: Term 1 election: Requested vote from peers e3a0aa52a3cc41489dd8d20149971578 (127.10.24.130:38253), 7f14b896f4e94cd58c98621ff06f0242 (127.10.24.129:39205)
I20250904 19:58:43.979959 15525 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "63602651692d46ac96799a909542ec00" candidate_uuid: "3c867a82454b4215a6176c32909df17d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f14b896f4e94cd58c98621ff06f0242"
I20250904 19:58:43.980578 15525 raft_consensus.cc:3058] T 63602651692d46ac96799a909542ec00 P 7f14b896f4e94cd58c98621ff06f0242 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:43.987198 15525 raft_consensus.cc:2466] T 63602651692d46ac96799a909542ec00 P 7f14b896f4e94cd58c98621ff06f0242 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c867a82454b4215a6176c32909df17d in term 1.
I20250904 19:58:43.988241 15646 leader_election.cc:304] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c867a82454b4215a6176c32909df17d, 7f14b896f4e94cd58c98621ff06f0242; no voters:
I20250904 19:58:43.989064 15743 raft_consensus.cc:2802] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:43.992894 15758 raft_consensus.cc:491] T 63602651692d46ac96799a909542ec00 P e3a0aa52a3cc41489dd8d20149971578 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:43.993430 15603 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "0591d48fddb94304bbcf561723dd0e95" candidate_uuid: "3c867a82454b4215a6176c32909df17d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3a0aa52a3cc41489dd8d20149971578" is_pre_election: true
I20250904 19:58:43.993389 15758 raft_consensus.cc:513] T 63602651692d46ac96799a909542ec00 P e3a0aa52a3cc41489dd8d20149971578 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } } peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } }
I20250904 19:58:43.994168 15603 raft_consensus.cc:2466] T 0591d48fddb94304bbcf561723dd0e95 P e3a0aa52a3cc41489dd8d20149971578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c867a82454b4215a6176c32909df17d in term 0.
I20250904 19:58:43.996568 15692 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "63602651692d46ac96799a909542ec00" candidate_uuid: "e3a0aa52a3cc41489dd8d20149971578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c867a82454b4215a6176c32909df17d" is_pre_election: true
I20250904 19:58:43.998340 15758 leader_election.cc:290] T 63602651692d46ac96799a909542ec00 P e3a0aa52a3cc41489dd8d20149971578 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c867a82454b4215a6176c32909df17d (127.10.24.131:33469), 7f14b896f4e94cd58c98621ff06f0242 (127.10.24.129:39205)
I20250904 19:58:43.999042 15603 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "63602651692d46ac96799a909542ec00" candidate_uuid: "3c867a82454b4215a6176c32909df17d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3a0aa52a3cc41489dd8d20149971578" is_pre_election: true
I20250904 19:58:43.999755 15603 raft_consensus.cc:2466] T 63602651692d46ac96799a909542ec00 P e3a0aa52a3cc41489dd8d20149971578 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c867a82454b4215a6176c32909df17d in term 0.
I20250904 19:58:44.000370 15741 tablet_bootstrap.cc:492] T 0591d48fddb94304bbcf561723dd0e95 P 7f14b896f4e94cd58c98621ff06f0242: No bootstrap required, opened a new log
I20250904 19:58:44.001037 15741 ts_tablet_manager.cc:1397] T 0591d48fddb94304bbcf561723dd0e95 P 7f14b896f4e94cd58c98621ff06f0242: Time spent bootstrapping tablet: real 0.079s user 0.016s sys 0.014s
I20250904 19:58:44.001189 15646 leader_election.cc:304] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 3c867a82454b4215a6176c32909df17d, e3a0aa52a3cc41489dd8d20149971578; no voters: 7f14b896f4e94cd58c98621ff06f0242
I20250904 19:58:44.002462 15603 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "63602651692d46ac96799a909542ec00" candidate_uuid: "3c867a82454b4215a6176c32909df17d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3a0aa52a3cc41489dd8d20149971578"
I20250904 19:58:44.005406 15741 raft_consensus.cc:357] T 0591d48fddb94304bbcf561723dd0e95 P 7f14b896f4e94cd58c98621ff06f0242 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } } peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } }
I20250904 19:58:44.006510 15741 raft_consensus.cc:383] T 0591d48fddb94304bbcf561723dd0e95 P 7f14b896f4e94cd58c98621ff06f0242 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:44.006794 15741 raft_consensus.cc:738] T 0591d48fddb94304bbcf561723dd0e95 P 7f14b896f4e94cd58c98621ff06f0242 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f14b896f4e94cd58c98621ff06f0242, State: Initialized, Role: FOLLOWER
I20250904 19:58:44.007270 15603 raft_consensus.cc:3058] T 63602651692d46ac96799a909542ec00 P e3a0aa52a3cc41489dd8d20149971578 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:44.007438 15741 consensus_queue.cc:260] T 0591d48fddb94304bbcf561723dd0e95 P 7f14b896f4e94cd58c98621ff06f0242 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } } peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } }
I20250904 19:58:44.011742 15741 ts_tablet_manager.cc:1428] T 0591d48fddb94304bbcf561723dd0e95 P 7f14b896f4e94cd58c98621ff06f0242: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20250904 19:58:44.028340 15759 raft_consensus.cc:2802] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:44.028781 15759 raft_consensus.cc:491] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:44.027758 15743 raft_consensus.cc:695] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d [term 1 LEADER]: Becoming Leader. State: Replica: 3c867a82454b4215a6176c32909df17d, State: Running, Role: LEADER
I20250904 19:58:44.029508 15525 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "63602651692d46ac96799a909542ec00" candidate_uuid: "e3a0aa52a3cc41489dd8d20149971578" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f14b896f4e94cd58c98621ff06f0242" is_pre_election: true
I20250904 19:58:44.030278 15525 raft_consensus.cc:2391] T 63602651692d46ac96799a909542ec00 P 7f14b896f4e94cd58c98621ff06f0242 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e3a0aa52a3cc41489dd8d20149971578 in current term 1: Already voted for candidate 3c867a82454b4215a6176c32909df17d in this term.
I20250904 19:58:44.029974 15743 consensus_queue.cc:237] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } } peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } }
I20250904 19:58:44.032855 15603 raft_consensus.cc:2466] T 63602651692d46ac96799a909542ec00 P e3a0aa52a3cc41489dd8d20149971578 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c867a82454b4215a6176c32909df17d in term 1.
I20250904 19:58:44.029106 15759 raft_consensus.cc:3058] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:44.037163 15269 catalog_manager.cc:5582] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3a0aa52a3cc41489dd8d20149971578 (127.10.24.130). New cstate: current_term: 1 leader_uuid: "e3a0aa52a3cc41489dd8d20149971578" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } health_report { overall_health: UNKNOWN } } }
I20250904 19:58:44.047376 15568 leader_election.cc:304] T 63602651692d46ac96799a909542ec00 P e3a0aa52a3cc41489dd8d20149971578 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e3a0aa52a3cc41489dd8d20149971578; no voters: 3c867a82454b4215a6176c32909df17d, 7f14b896f4e94cd58c98621ff06f0242
I20250904 19:58:44.048327 15745 raft_consensus.cc:2747] T 63602651692d46ac96799a909542ec00 P e3a0aa52a3cc41489dd8d20149971578 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250904 19:58:44.052832 15268 catalog_manager.cc:5582] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c867a82454b4215a6176c32909df17d (127.10.24.131). New cstate: current_term: 1 leader_uuid: "3c867a82454b4215a6176c32909df17d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } health_report { overall_health: UNKNOWN } } }
I20250904 19:58:44.058471 15759 raft_consensus.cc:513] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } } peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } }
I20250904 19:58:44.060549 15759 leader_election.cc:290] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [CANDIDATE]: Term 1 election: Requested vote from peers 7f14b896f4e94cd58c98621ff06f0242 (127.10.24.129:39205), e3a0aa52a3cc41489dd8d20149971578 (127.10.24.130:38253)
I20250904 19:58:44.061650 15525 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "0591d48fddb94304bbcf561723dd0e95" candidate_uuid: "3c867a82454b4215a6176c32909df17d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f14b896f4e94cd58c98621ff06f0242"
I20250904 19:58:44.062065 15603 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "0591d48fddb94304bbcf561723dd0e95" candidate_uuid: "3c867a82454b4215a6176c32909df17d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3a0aa52a3cc41489dd8d20149971578"
I20250904 19:58:44.062403 15525 raft_consensus.cc:3058] T 0591d48fddb94304bbcf561723dd0e95 P 7f14b896f4e94cd58c98621ff06f0242 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:44.062600 15603 raft_consensus.cc:3058] T 0591d48fddb94304bbcf561723dd0e95 P e3a0aa52a3cc41489dd8d20149971578 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:44.069972 15525 raft_consensus.cc:2466] T 0591d48fddb94304bbcf561723dd0e95 P 7f14b896f4e94cd58c98621ff06f0242 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c867a82454b4215a6176c32909df17d in term 1.
I20250904 19:58:44.071877 15646 leader_election.cc:304] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c867a82454b4215a6176c32909df17d, 7f14b896f4e94cd58c98621ff06f0242; no voters:
I20250904 19:58:44.078011 15603 raft_consensus.cc:2466] T 0591d48fddb94304bbcf561723dd0e95 P e3a0aa52a3cc41489dd8d20149971578 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c867a82454b4215a6176c32909df17d in term 1.
I20250904 19:58:44.080998 15759 raft_consensus.cc:2802] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:44.081627 15759 raft_consensus.cc:695] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [term 1 LEADER]: Becoming Leader. State: Replica: 3c867a82454b4215a6176c32909df17d, State: Running, Role: LEADER
I20250904 19:58:44.082530 15759 consensus_queue.cc:237] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } } peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } }
I20250904 19:58:44.137528 15269 catalog_manager.cc:5582] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c867a82454b4215a6176c32909df17d (127.10.24.131). New cstate: current_term: 1 leader_uuid: "3c867a82454b4215a6176c32909df17d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f14b896f4e94cd58c98621ff06f0242" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 39205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e3a0aa52a3cc41489dd8d20149971578" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 38253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 } health_report { overall_health: HEALTHY } } }
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tools/ksck_remote-test.cc:514: Skipped
test is skipped; set KUDU_ALLOW_SLOW_TESTS=1 to run
I20250904 19:58:44.256137 10338 tablet_server.cc:178] TabletServer@127.10.24.129:0 shutting down...
I20250904 19:58:44.302291 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:44.303164 10338 tablet_replica.cc:331] T 0591d48fddb94304bbcf561723dd0e95 P 7f14b896f4e94cd58c98621ff06f0242: stopping tablet replica
I20250904 19:58:44.303829 10338 raft_consensus.cc:2241] T 0591d48fddb94304bbcf561723dd0e95 P 7f14b896f4e94cd58c98621ff06f0242 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:44.304404 10338 raft_consensus.cc:2270] T 0591d48fddb94304bbcf561723dd0e95 P 7f14b896f4e94cd58c98621ff06f0242 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:44.306524 10338 tablet_replica.cc:331] T 6b808d2abc454da28f9a53ce9d46b3e5 P 7f14b896f4e94cd58c98621ff06f0242: stopping tablet replica
I20250904 19:58:44.307258 10338 raft_consensus.cc:2241] T 6b808d2abc454da28f9a53ce9d46b3e5 P 7f14b896f4e94cd58c98621ff06f0242 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:44.307760 10338 raft_consensus.cc:2270] T 6b808d2abc454da28f9a53ce9d46b3e5 P 7f14b896f4e94cd58c98621ff06f0242 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:44.313377 10338 tablet_replica.cc:331] T 63602651692d46ac96799a909542ec00 P 7f14b896f4e94cd58c98621ff06f0242: stopping tablet replica
I20250904 19:58:44.313855 10338 raft_consensus.cc:2241] T 63602651692d46ac96799a909542ec00 P 7f14b896f4e94cd58c98621ff06f0242 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:44.314199 10338 raft_consensus.cc:2270] T 63602651692d46ac96799a909542ec00 P 7f14b896f4e94cd58c98621ff06f0242 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:44.336417 10338 tablet_server.cc:195] TabletServer@127.10.24.129:0 shutdown complete.
I20250904 19:58:44.349859 10338 tablet_server.cc:178] TabletServer@127.10.24.130:0 shutting down...
I20250904 19:58:44.378118 15745 consensus_queue.cc:1035] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c867a82454b4215a6176c32909df17d" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 33469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:44.382915 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:44.384539 10338 tablet_replica.cc:331] T 0591d48fddb94304bbcf561723dd0e95 P e3a0aa52a3cc41489dd8d20149971578: stopping tablet replica
I20250904 19:58:44.385324 10338 raft_consensus.cc:2241] T 0591d48fddb94304bbcf561723dd0e95 P e3a0aa52a3cc41489dd8d20149971578 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:44.385862 10338 raft_consensus.cc:2270] T 0591d48fddb94304bbcf561723dd0e95 P e3a0aa52a3cc41489dd8d20149971578 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:44.388084 10338 tablet_replica.cc:331] T 63602651692d46ac96799a909542ec00 P e3a0aa52a3cc41489dd8d20149971578: stopping tablet replica
I20250904 19:58:44.388628 10338 raft_consensus.cc:2241] T 63602651692d46ac96799a909542ec00 P e3a0aa52a3cc41489dd8d20149971578 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:44.388996 10338 raft_consensus.cc:2270] T 63602651692d46ac96799a909542ec00 P e3a0aa52a3cc41489dd8d20149971578 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:44.392278 10338 tablet_replica.cc:331] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578: stopping tablet replica
I20250904 19:58:44.392979 10338 raft_consensus.cc:2241] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:44.393749 10338 raft_consensus.cc:2270] T 6b808d2abc454da28f9a53ce9d46b3e5 P e3a0aa52a3cc41489dd8d20149971578 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:44.414206 10338 tablet_server.cc:195] TabletServer@127.10.24.130:0 shutdown complete.
I20250904 19:58:44.424842 10338 tablet_server.cc:178] TabletServer@127.10.24.131:0 shutting down...
I20250904 19:58:44.446758 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:44.447310 10338 tablet_replica.cc:331] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d: stopping tablet replica
I20250904 19:58:44.447867 10338 raft_consensus.cc:2241] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:44.448436 10338 pending_rounds.cc:70] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d: Trying to abort 1 pending ops.
I20250904 19:58:44.448635 10338 pending_rounds.cc:77] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7196737225106575360 op_type: NO_OP noop_request { }
I20250904 19:58:44.448901 10338 raft_consensus.cc:2887] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250904 19:58:44.449160 10338 raft_consensus.cc:2270] T 0591d48fddb94304bbcf561723dd0e95 P 3c867a82454b4215a6176c32909df17d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:44.451110 10338 tablet_replica.cc:331] T 6b808d2abc454da28f9a53ce9d46b3e5 P 3c867a82454b4215a6176c32909df17d: stopping tablet replica
I20250904 19:58:44.451628 10338 raft_consensus.cc:2241] T 6b808d2abc454da28f9a53ce9d46b3e5 P 3c867a82454b4215a6176c32909df17d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:44.452082 10338 raft_consensus.cc:2270] T 6b808d2abc454da28f9a53ce9d46b3e5 P 3c867a82454b4215a6176c32909df17d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:44.454039 10338 tablet_replica.cc:331] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d: stopping tablet replica
I20250904 19:58:44.454476 10338 raft_consensus.cc:2241] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:44.455013 10338 pending_rounds.cc:70] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d: Trying to abort 1 pending ops.
I20250904 19:58:44.455179 10338 pending_rounds.cc:77] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7196737224839884800 op_type: NO_OP noop_request { }
I20250904 19:58:44.455416 10338 raft_consensus.cc:2887] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250904 19:58:44.455703 10338 raft_consensus.cc:2270] T 63602651692d46ac96799a909542ec00 P 3c867a82454b4215a6176c32909df17d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:44.468032 10338 tablet_server.cc:195] TabletServer@127.10.24.131:0 shutdown complete.
I20250904 19:58:44.478065 10338 master.cc:561] Master@127.10.24.190:39577 shutting down...
I20250904 19:58:44.492163 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:44.492957 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:44.493248 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P af299e90d7cc4c9180cead65dedf1298: stopping tablet replica
I20250904 19:58:44.511878 10338 master.cc:583] Master@127.10.24.190:39577 shutdown complete.
I20250904 19:58:44.523187 10338 master.cc:561] Master@127.10.24.189:40799 shutting down...
I20250904 19:58:44.535101 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:44.535622 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:44.535902 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 65dd321cbb564039bb2b879cf12f82cf: stopping tablet replica
I20250904 19:58:44.554689 10338 master.cc:583] Master@127.10.24.189:40799 shutdown complete.
I20250904 19:58:44.564527 10338 master.cc:561] Master@127.10.24.188:41125 shutting down...
I20250904 19:58:44.577481 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:44.577987 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:44.578241 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 66dff7d2052c487da0691f348efce8fb: stopping tablet replica
I20250904 19:58:44.596598 10338 master.cc:583] Master@127.10.24.188:41125 shutdown complete.
[ SKIPPED ] RemoteKsckTest.TestChecksumSnapshotLastingLongerThanAHM (2257 ms)
[ RUN ] RemoteKsckTest.TestLeaderMasterDown
I20250904 19:58:44.620792 10338 test_util.cc:276] Using random seed: 638402016
I20250904 19:58:44.621662 10338 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.24.190:33243,127.10.24.189:38645,127.10.24.188:39667
I20250904 19:58:44.622840 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:44.627522 15779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:44.627717 15780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:44.628556 15782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:44.629551 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:44.630245 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:44.630434 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:44.630566 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015924630551 us; error 0 us; skew 500 ppm
I20250904 19:58:44.631086 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:44.633128 10338 webserver.cc:480] Webserver started at http://127.10.24.190:46737/ using document root <none> and password file <none>
I20250904 19:58:44.633565 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:44.633728 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:44.633949 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:44.634907 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/master-0-root/instance:
uuid: "8f10ffaee87a4bd8b48b60ca5adad90c"
format_stamp: "Formatted at 2025-09-04 19:58:44 on dist-test-slave-f3xj"
I20250904 19:58:44.639055 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250904 19:58:44.642087 15787 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:44.642717 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250904 19:58:44.642964 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/master-0-root
uuid: "8f10ffaee87a4bd8b48b60ca5adad90c"
format_stamp: "Formatted at 2025-09-04 19:58:44 on dist-test-slave-f3xj"
I20250904 19:58:44.643213 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:44.666340 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:44.667263 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:44.699657 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.190:33243
I20250904 19:58:44.699753 15838 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.190:33243 every 8 connection(s)
I20250904 19:58:44.703227 15839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:44.703450 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:44.708576 15841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:44.709084 15842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:44.710726 15839 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:58:44.711297 15844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:44.711357 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:44.712703 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:44.712977 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:44.713162 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015924713148 us; error 0 us; skew 500 ppm
I20250904 19:58:44.713827 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:44.716439 10338 webserver.cc:480] Webserver started at http://127.10.24.189:34237/ using document root <none> and password file <none>
I20250904 19:58:44.717048 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:44.717278 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:44.717563 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:44.718683 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/master-1-root/instance:
uuid: "e1e217baccf24181889482aa9a783e94"
format_stamp: "Formatted at 2025-09-04 19:58:44 on dist-test-slave-f3xj"
I20250904 19:58:44.723181 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20250904 19:58:44.726426 15851 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:44.727114 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250904 19:58:44.727388 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/master-1-root
uuid: "e1e217baccf24181889482aa9a783e94"
format_stamp: "Formatted at 2025-09-04 19:58:44 on dist-test-slave-f3xj"
I20250904 19:58:44.727715 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:44.731180 15839 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:58:44.736680 15839 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.189:38645: Network error: Client connection negotiation failed: client connection to 127.10.24.189:38645: connect: Connection refused (error 111)
I20250904 19:58:44.739209 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:44.740211 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:44.774042 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.189:38645
I20250904 19:58:44.774166 15903 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.189:38645 every 8 connection(s)
I20250904 19:58:44.777678 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:44.777781 15904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20250904 19:58:44.782831 15906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:44.783217 15907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:44.785763 15909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:44.786945 15904 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:44.790016 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:44.790958 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:44.791215 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:44.791411 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015924791391 us; error 0 us; skew 500 ppm
I20250904 19:58:44.791982 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:44.794507 10338 webserver.cc:480] Webserver started at http://127.10.24.188:35111/ using document root <none> and password file <none>
I20250904 19:58:44.795112 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:44.795351 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:44.795667 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:44.796943 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/master-2-root/instance:
uuid: "d5ee58d0539642938a4acad2774191aa"
format_stamp: "Formatted at 2025-09-04 19:58:44 on dist-test-slave-f3xj"
I20250904 19:58:44.798440 15904 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:44.800103 15839 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } attempt: 1
I20250904 19:58:44.804919 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20250904 19:58:44.809743 15917 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:44.810825 10338 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20250904 19:58:44.811169 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/master-2-root
uuid: "d5ee58d0539642938a4acad2774191aa"
format_stamp: "Formatted at 2025-09-04 19:58:44 on dist-test-slave-f3xj"
I20250904 19:58:44.811497 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:44.811904 15904 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:44.812250 15839 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:58:44.816825 15839 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:39667: Network error: Client connection negotiation failed: client connection to 127.10.24.188:39667: connect: Connection refused (error 111)
W20250904 19:58:44.817894 15904 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:39667: Network error: Client connection negotiation failed: client connection to 127.10.24.188:39667: connect: Connection refused (error 111)
I20250904 19:58:44.825933 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:44.827380 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:44.845351 15839 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } attempt: 1
W20250904 19:58:44.849949 15839 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:39667: Network error: Client connection negotiation failed: client connection to 127.10.24.188:39667: connect: Connection refused (error 111)
I20250904 19:58:44.851395 15904 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } attempt: 1
W20250904 19:58:44.856678 15904 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:39667: Network error: Client connection negotiation failed: client connection to 127.10.24.188:39667: connect: Connection refused (error 111)
I20250904 19:58:44.866951 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.188:39667
I20250904 19:58:44.867045 15969 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.188:39667 every 8 connection(s)
I20250904 19:58:44.869859 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 19:58:44.870739 15970 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:44.876695 15970 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:44.883445 15839 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } attempt: 2
I20250904 19:58:44.886961 15970 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:44.897001 15970 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:44.897192 15904 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } attempt: 2
I20250904 19:58:44.901746 15839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c: Bootstrap starting.
I20250904 19:58:44.908617 15839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:44.915009 15970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa: Bootstrap starting.
I20250904 19:58:44.917850 15839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c: No bootstrap required, opened a new log
I20250904 19:58:44.919781 15970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:44.920625 15839 raft_consensus.cc:357] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } }
I20250904 19:58:44.921330 15839 raft_consensus.cc:383] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:44.921609 15839 raft_consensus.cc:738] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f10ffaee87a4bd8b48b60ca5adad90c, State: Initialized, Role: FOLLOWER
I20250904 19:58:44.922318 15839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } }
I20250904 19:58:44.923053 15904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94: Bootstrap starting.
I20250904 19:58:44.924062 15970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa: No bootstrap required, opened a new log
I20250904 19:58:44.924400 15977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } } }
I20250904 19:58:44.925213 15977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:44.926069 15839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:44.926451 15970 raft_consensus.cc:357] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } }
I20250904 19:58:44.926965 15970 raft_consensus.cc:383] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:44.927174 15970 raft_consensus.cc:738] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5ee58d0539642938a4acad2774191aa, State: Initialized, Role: FOLLOWER
I20250904 19:58:44.927776 15970 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } }
I20250904 19:58:44.929105 15904 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:44.929862 15978 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } } }
I20250904 19:58:44.930575 15978 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:44.931427 15970 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:44.933739 15904 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94: No bootstrap required, opened a new log
I20250904 19:58:44.936249 15904 raft_consensus.cc:357] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } }
I20250904 19:58:44.936920 15904 raft_consensus.cc:383] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:44.937186 15904 raft_consensus.cc:738] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1e217baccf24181889482aa9a783e94, State: Initialized, Role: FOLLOWER
I20250904 19:58:44.937930 15904 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } }
I20250904 19:58:44.940362 15992 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } } }
I20250904 19:58:44.941087 15992 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:44.941906 15904 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:44.958220 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20250904 19:58:44.959831 16012 catalog_manager.cc:797] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20250904 19:58:44.962371 16004 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:58:44.962633 16004 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20250904 19:58:44.962774 16013 catalog_manager.cc:1560] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20250904 19:58:44.962913 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
W20250904 19:58:44.963101 16013 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250904 19:58:44.964286 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:44.970650 16014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:44.970752 16015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:44.971839 16017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:44.972676 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:44.973446 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:44.973625 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:44.973752 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015924973740 us; error 0 us; skew 500 ppm
I20250904 19:58:44.974196 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:44.976338 10338 webserver.cc:480] Webserver started at http://127.10.24.129:39247/ using document root <none> and password file <none>
I20250904 19:58:44.976765 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:44.976909 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:44.977109 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:44.978080 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/ts-0-root/instance:
uuid: "2a69af64672e4feb9c8d03702b501642"
format_stamp: "Formatted at 2025-09-04 19:58:44 on dist-test-slave-f3xj"
I20250904 19:58:44.982108 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250904 19:58:44.985220 16022 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:44.985927 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20250904 19:58:44.986173 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/ts-0-root
uuid: "2a69af64672e4feb9c8d03702b501642"
format_stamp: "Formatted at 2025-09-04 19:58:44 on dist-test-slave-f3xj"
I20250904 19:58:44.986428 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:45.006700 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:45.007804 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:45.009117 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:45.011260 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:45.011443 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:45.011691 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:45.011837 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:45.048419 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.129:34527
I20250904 19:58:45.048534 16084 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.129:34527 every 8 connection(s)
I20250904 19:58:45.069270 16085 heartbeater.cc:344] Connected to a master server at 127.10.24.189:38645
I20250904 19:58:45.069682 16085 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:45.070533 16085 heartbeater.cc:507] Master 127.10.24.189:38645 requested a full tablet report, sending...
I20250904 19:58:45.071337 15977 raft_consensus.cc:491] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:45.071892 15977 raft_consensus.cc:513] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } }
I20250904 19:58:45.073454 15869 ts_manager.cc:194] Registered new tserver with Master: 2a69af64672e4feb9c8d03702b501642 (127.10.24.129:34527)
I20250904 19:58:45.074007 15977 leader_election.cc:290] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e1e217baccf24181889482aa9a783e94 (127.10.24.189:38645), d5ee58d0539642938a4acad2774191aa (127.10.24.188:39667)
I20250904 19:58:45.076179 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:45.076145 15945 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5ee58d0539642938a4acad2774191aa" is_pre_election: true
I20250904 19:58:45.076953 15945 raft_consensus.cc:2466] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f10ffaee87a4bd8b48b60ca5adad90c in term 0.
I20250904 19:58:45.076938 15879 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1e217baccf24181889482aa9a783e94" is_pre_election: true
I20250904 19:58:45.078199 15879 raft_consensus.cc:2466] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f10ffaee87a4bd8b48b60ca5adad90c in term 0.
I20250904 19:58:45.078315 15789 leader_election.cc:304] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f10ffaee87a4bd8b48b60ca5adad90c, d5ee58d0539642938a4acad2774191aa; no voters:
I20250904 19:58:45.079252 16087 heartbeater.cc:344] Connected to a master server at 127.10.24.190:33243
I20250904 19:58:45.079648 16087 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:45.080317 15977 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:45.080502 16087 heartbeater.cc:507] Master 127.10.24.190:33243 requested a full tablet report, sending...
I20250904 19:58:45.080751 15977 raft_consensus.cc:491] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:45.081254 15977 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:45.082435 16086 heartbeater.cc:344] Connected to a master server at 127.10.24.188:39667
I20250904 19:58:45.082854 16086 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:45.083750 16086 heartbeater.cc:507] Master 127.10.24.188:39667 requested a full tablet report, sending...
I20250904 19:58:45.086607 15935 ts_manager.cc:194] Registered new tserver with Master: 2a69af64672e4feb9c8d03702b501642 (127.10.24.129:34527)
W20250904 19:58:45.087039 16092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:45.087776 15977 raft_consensus.cc:513] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } }
I20250904 19:58:45.089597 15977 leader_election.cc:290] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [CANDIDATE]: Term 1 election: Requested vote from peers e1e217baccf24181889482aa9a783e94 (127.10.24.189:38645), d5ee58d0539642938a4acad2774191aa (127.10.24.188:39667)
W20250904 19:58:45.089692 16093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:45.090633 15879 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1e217baccf24181889482aa9a783e94"
I20250904 19:58:45.091089 15945 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5ee58d0539642938a4acad2774191aa"
I20250904 19:58:45.091360 15804 ts_manager.cc:194] Registered new tserver with Master: 2a69af64672e4feb9c8d03702b501642 (127.10.24.129:34527)
I20250904 19:58:45.091180 15879 raft_consensus.cc:3058] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:45.091789 15945 raft_consensus.cc:3058] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:45.099383 15879 raft_consensus.cc:2466] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f10ffaee87a4bd8b48b60ca5adad90c in term 1.
I20250904 19:58:45.100703 15789 leader_election.cc:304] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f10ffaee87a4bd8b48b60ca5adad90c, e1e217baccf24181889482aa9a783e94; no voters:
I20250904 19:58:45.101244 15945 raft_consensus.cc:2466] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f10ffaee87a4bd8b48b60ca5adad90c in term 1.
I20250904 19:58:45.101686 15977 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:45.103102 15977 raft_consensus.cc:695] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [term 1 LEADER]: Becoming Leader. State: Replica: 8f10ffaee87a4bd8b48b60ca5adad90c, State: Running, Role: LEADER
I20250904 19:58:45.104002 15977 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } }
I20250904 19:58:45.112396 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:45.112488 16097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f10ffaee87a4bd8b48b60ca5adad90c. Latest consensus state: current_term: 1 leader_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } } }
W20250904 19:58:45.112337 16095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:45.113421 16097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:45.114094 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:45.114334 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:45.114516 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015925114497 us; error 0 us; skew 500 ppm
I20250904 19:58:45.114817 16099 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 19:58:45.115105 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:45.120090 16099 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 19:58:45.121407 10338 webserver.cc:480] Webserver started at http://127.10.24.130:37851/ using document root <none> and password file <none>
I20250904 19:58:45.121917 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:45.122100 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:45.122344 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:45.123504 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/ts-1-root/instance:
uuid: "3a083c1fe3c44686837e39e67696567f"
format_stamp: "Formatted at 2025-09-04 19:58:45 on dist-test-slave-f3xj"
I20250904 19:58:45.128083 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20250904 19:58:45.132117 15945 raft_consensus.cc:1273] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [term 1 FOLLOWER]: Refusing update from remote peer 8f10ffaee87a4bd8b48b60ca5adad90c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:45.132470 15879 raft_consensus.cc:1273] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [term 1 FOLLOWER]: Refusing update from remote peer 8f10ffaee87a4bd8b48b60ca5adad90c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:45.134270 15977 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:45.134860 16104 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:45.135988 10338 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.006s sys 0.001s
I20250904 19:58:45.136314 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/ts-1-root
uuid: "3a083c1fe3c44686837e39e67696567f"
format_stamp: "Formatted at 2025-09-04 19:58:45 on dist-test-slave-f3xj"
I20250904 19:58:45.136654 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:45.138109 15977 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:45.143782 15992 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f10ffaee87a4bd8b48b60ca5adad90c. Latest consensus state: current_term: 1 leader_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } } }
I20250904 19:58:45.144510 15992 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:45.155838 15978 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f10ffaee87a4bd8b48b60ca5adad90c. Latest consensus state: current_term: 1 leader_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } } }
I20250904 19:58:45.156596 15978 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:45.161194 16097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } } }
I20250904 19:58:45.161890 16097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:45.167439 15977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } } }
I20250904 19:58:45.169196 15977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:45.174288 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:45.175671 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:45.177955 16099 catalog_manager.cc:1349] Generated new cluster ID: 69ebf69582804f788cbebb9f2b2f841d
I20250904 19:58:45.178288 16099 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 19:58:45.179940 15992 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } } }
I20250904 19:58:45.180732 15992 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:45.187161 15978 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } } }
I20250904 19:58:45.187940 15978 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:45.192211 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:45.194538 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:45.194706 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:45.194942 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:45.195093 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:45.225090 16099 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 19:58:45.226768 16099 catalog_manager.cc:1506] Loading token signing keys...
I20250904 19:58:45.255853 16099 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c: Generated new TSK 0
I20250904 19:58:45.256559 16099 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 19:58:45.264367 15804 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40488
I20250904 19:58:45.276783 16087 heartbeater.cc:499] Master 127.10.24.190:33243 was elected leader, sending a full tablet report...
I20250904 19:58:45.283443 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.130:43963
I20250904 19:58:45.283531 16175 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.130:43963 every 8 connection(s)
I20250904 19:58:45.295269 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:45.309340 16176 heartbeater.cc:344] Connected to a master server at 127.10.24.189:38645
I20250904 19:58:45.309765 16176 heartbeater.cc:461] Registering TS with master...
W20250904 19:58:45.310148 16183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:45.310591 16176 heartbeater.cc:507] Master 127.10.24.189:38645 requested a full tablet report, sending...
I20250904 19:58:45.315227 15868 ts_manager.cc:194] Registered new tserver with Master: 3a083c1fe3c44686837e39e67696567f (127.10.24.130:43963)
I20250904 19:58:45.318608 16177 heartbeater.cc:344] Connected to a master server at 127.10.24.188:39667
I20250904 19:58:45.318941 16177 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:45.319885 16177 heartbeater.cc:507] Master 127.10.24.188:39667 requested a full tablet report, sending...
I20250904 19:58:45.322449 15935 ts_manager.cc:194] Registered new tserver with Master: 3a083c1fe3c44686837e39e67696567f (127.10.24.130:43963)
W20250904 19:58:45.324443 16184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:45.324679 16179 heartbeater.cc:344] Connected to a master server at 127.10.24.190:33243
I20250904 19:58:45.325146 16179 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:45.325635 10338 server_base.cc:1047] running on GCE node
W20250904 19:58:45.325896 16187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:45.325973 16179 heartbeater.cc:507] Master 127.10.24.190:33243 requested a full tablet report, sending...
I20250904 19:58:45.327019 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:45.327318 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:45.327503 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015925327488 us; error 0 us; skew 500 ppm
I20250904 19:58:45.328152 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:45.328145 15804 ts_manager.cc:194] Registered new tserver with Master: 3a083c1fe3c44686837e39e67696567f (127.10.24.130:43963)
I20250904 19:58:45.329850 15804 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40500
I20250904 19:58:45.331420 10338 webserver.cc:480] Webserver started at http://127.10.24.131:42101/ using document root <none> and password file <none>
I20250904 19:58:45.332144 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:45.332392 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:45.332707 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:45.334825 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/ts-2-root/instance:
uuid: "01a51f25e351493b85dbd79ae7b2e3ca"
format_stamp: "Formatted at 2025-09-04 19:58:45 on dist-test-slave-f3xj"
I20250904 19:58:45.339884 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20250904 19:58:45.342662 16179 heartbeater.cc:499] Master 127.10.24.190:33243 was elected leader, sending a full tablet report...
I20250904 19:58:45.342821 16191 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:45.343710 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250904 19:58:45.344046 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/ts-2-root
uuid: "01a51f25e351493b85dbd79ae7b2e3ca"
format_stamp: "Formatted at 2025-09-04 19:58:45 on dist-test-slave-f3xj"
I20250904 19:58:45.344358 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestLeaderMasterDown.1757015869512910-10338-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:45.359373 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:45.360478 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:45.361937 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:45.365374 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:45.365593 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:45.365825 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:45.366040 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:45.425529 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.131:37413
I20250904 19:58:45.425628 16253 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.131:37413 every 8 connection(s)
I20250904 19:58:45.449694 16254 heartbeater.cc:344] Connected to a master server at 127.10.24.189:38645
I20250904 19:58:45.450134 16254 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:45.450978 16254 heartbeater.cc:507] Master 127.10.24.189:38645 requested a full tablet report, sending...
I20250904 19:58:45.453547 15868 ts_manager.cc:194] Registered new tserver with Master: 01a51f25e351493b85dbd79ae7b2e3ca (127.10.24.131:37413)
I20250904 19:58:45.458320 16255 heartbeater.cc:344] Connected to a master server at 127.10.24.188:39667
I20250904 19:58:45.458792 16255 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:45.459748 16255 heartbeater.cc:507] Master 127.10.24.188:39667 requested a full tablet report, sending...
I20250904 19:58:45.460489 16256 heartbeater.cc:344] Connected to a master server at 127.10.24.190:33243
I20250904 19:58:45.461127 16256 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:45.461867 16256 heartbeater.cc:507] Master 127.10.24.190:33243 requested a full tablet report, sending...
I20250904 19:58:45.462074 15934 ts_manager.cc:194] Registered new tserver with Master: 01a51f25e351493b85dbd79ae7b2e3ca (127.10.24.131:37413)
I20250904 19:58:45.463970 15803 ts_manager.cc:194] Registered new tserver with Master: 01a51f25e351493b85dbd79ae7b2e3ca (127.10.24.131:37413)
I20250904 19:58:45.464278 10338 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013334789s
I20250904 19:58:45.465495 15803 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40510
I20250904 19:58:45.481305 16256 heartbeater.cc:499] Master 127.10.24.190:33243 was elected leader, sending a full tablet report...
I20250904 19:58:45.516947 15804 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40518:
name: "ksck-test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\n\000\000\000\004\001\024\000\000\000""\004\001\n\000\000\000\004\001\024\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 19:58:45.519841 15804 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table ksck-test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 19:58:45.626600 16140 tablet_service.cc:1468] Processing CreateTablet for tablet ae3869954dfa4f7db43da6d34dcefa6c (DEFAULT_TABLE table=ksck-test-table [id=8a653c26d4bc4f7ebef7bb6fd3623fb9]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:45.626776 16141 tablet_service.cc:1468] Processing CreateTablet for tablet 6b1b6476b0cc4a218f04c9e927199dc6 (DEFAULT_TABLE table=ksck-test-table [id=8a653c26d4bc4f7ebef7bb6fd3623fb9]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:45.627801 16140 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae3869954dfa4f7db43da6d34dcefa6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:45.630888 16141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b1b6476b0cc4a218f04c9e927199dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:45.644309 16139 tablet_service.cc:1468] Processing CreateTablet for tablet 60b599c4af8b43a4b78b5721bf3b6dbd (DEFAULT_TABLE table=ksck-test-table [id=8a653c26d4bc4f7ebef7bb6fd3623fb9]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:45.645679 16139 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60b599c4af8b43a4b78b5721bf3b6dbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:45.666029 16275 tablet_bootstrap.cc:492] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f: Bootstrap starting.
I20250904 19:58:45.672048 16275 tablet_bootstrap.cc:654] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:45.684383 16275 tablet_bootstrap.cc:492] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f: No bootstrap required, opened a new log
I20250904 19:58:45.684877 16275 ts_tablet_manager.cc:1397] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.005s
I20250904 19:58:45.688253 16275 raft_consensus.cc:357] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:45.688962 16275 raft_consensus.cc:383] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:45.689230 16275 raft_consensus.cc:738] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a083c1fe3c44686837e39e67696567f, State: Initialized, Role: FOLLOWER
I20250904 19:58:45.689940 16275 consensus_queue.cc:260] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:45.693084 16275 ts_tablet_manager.cc:1428] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20250904 19:58:45.698974 16275 tablet_bootstrap.cc:492] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f: Bootstrap starting.
I20250904 19:58:45.704835 16275 tablet_bootstrap.cc:654] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:45.707655 16049 tablet_service.cc:1468] Processing CreateTablet for tablet ae3869954dfa4f7db43da6d34dcefa6c (DEFAULT_TABLE table=ksck-test-table [id=8a653c26d4bc4f7ebef7bb6fd3623fb9]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:45.708979 16049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae3869954dfa4f7db43da6d34dcefa6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:45.708712 16050 tablet_service.cc:1468] Processing CreateTablet for tablet 6b1b6476b0cc4a218f04c9e927199dc6 (DEFAULT_TABLE table=ksck-test-table [id=8a653c26d4bc4f7ebef7bb6fd3623fb9]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:45.709939 16050 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b1b6476b0cc4a218f04c9e927199dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:45.710692 16048 tablet_service.cc:1468] Processing CreateTablet for tablet 60b599c4af8b43a4b78b5721bf3b6dbd (DEFAULT_TABLE table=ksck-test-table [id=8a653c26d4bc4f7ebef7bb6fd3623fb9]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:45.711879 16048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60b599c4af8b43a4b78b5721bf3b6dbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:45.730834 16219 tablet_service.cc:1468] Processing CreateTablet for tablet 6b1b6476b0cc4a218f04c9e927199dc6 (DEFAULT_TABLE table=ksck-test-table [id=8a653c26d4bc4f7ebef7bb6fd3623fb9]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:45.732151 16219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b1b6476b0cc4a218f04c9e927199dc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:45.736438 16218 tablet_service.cc:1468] Processing CreateTablet for tablet ae3869954dfa4f7db43da6d34dcefa6c (DEFAULT_TABLE table=ksck-test-table [id=8a653c26d4bc4f7ebef7bb6fd3623fb9]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:45.737727 16218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae3869954dfa4f7db43da6d34dcefa6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:45.739019 16275 tablet_bootstrap.cc:492] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f: No bootstrap required, opened a new log
I20250904 19:58:45.739657 16275 ts_tablet_manager.cc:1397] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f: Time spent bootstrapping tablet: real 0.041s user 0.019s sys 0.019s
I20250904 19:58:45.740489 16217 tablet_service.cc:1468] Processing CreateTablet for tablet 60b599c4af8b43a4b78b5721bf3b6dbd (DEFAULT_TABLE table=ksck-test-table [id=8a653c26d4bc4f7ebef7bb6fd3623fb9]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:45.741812 16217 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60b599c4af8b43a4b78b5721bf3b6dbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:45.742417 16275 raft_consensus.cc:357] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:45.743152 16275 raft_consensus.cc:383] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:45.743441 16275 raft_consensus.cc:738] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a083c1fe3c44686837e39e67696567f, State: Initialized, Role: FOLLOWER
I20250904 19:58:45.744199 16275 consensus_queue.cc:260] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:45.746924 16275 ts_tablet_manager.cc:1428] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250904 19:58:45.767678 16275 tablet_bootstrap.cc:492] T 6b1b6476b0cc4a218f04c9e927199dc6 P 3a083c1fe3c44686837e39e67696567f: Bootstrap starting.
I20250904 19:58:45.773584 16275 tablet_bootstrap.cc:654] T 6b1b6476b0cc4a218f04c9e927199dc6 P 3a083c1fe3c44686837e39e67696567f: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:45.872856 16277 raft_consensus.cc:491] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:45.873358 16277 raft_consensus.cc:513] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:45.875770 16277 leader_election.cc:290] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a69af64672e4feb9c8d03702b501642 (127.10.24.129:34527), 01a51f25e351493b85dbd79ae7b2e3ca (127.10.24.131:37413)
I20250904 19:58:45.884172 16279 tablet_bootstrap.cc:492] T 6b1b6476b0cc4a218f04c9e927199dc6 P 2a69af64672e4feb9c8d03702b501642: Bootstrap starting.
W20250904 19:58:45.919770 16110 leader_election.cc:343] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01a51f25e351493b85dbd79ae7b2e3ca (127.10.24.131:37413): Not found: Tablet not found: ae3869954dfa4f7db43da6d34dcefa6c
W20250904 19:58:45.923122 16108 leader_election.cc:343] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a69af64672e4feb9c8d03702b501642 (127.10.24.129:34527): Not found: Tablet not found: ae3869954dfa4f7db43da6d34dcefa6c
I20250904 19:58:45.923743 16108 leader_election.cc:304] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a083c1fe3c44686837e39e67696567f; no voters: 01a51f25e351493b85dbd79ae7b2e3ca, 2a69af64672e4feb9c8d03702b501642
I20250904 19:58:45.924151 16279 tablet_bootstrap.cc:654] T 6b1b6476b0cc4a218f04c9e927199dc6 P 2a69af64672e4feb9c8d03702b501642: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:45.932143 16281 tablet_bootstrap.cc:492] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca: Bootstrap starting.
I20250904 19:58:45.933444 16277 raft_consensus.cc:2747] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250904 19:58:45.937036 16281 tablet_bootstrap.cc:654] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:45.947935 16275 tablet_bootstrap.cc:492] T 6b1b6476b0cc4a218f04c9e927199dc6 P 3a083c1fe3c44686837e39e67696567f: No bootstrap required, opened a new log
I20250904 19:58:45.948453 16275 ts_tablet_manager.cc:1397] T 6b1b6476b0cc4a218f04c9e927199dc6 P 3a083c1fe3c44686837e39e67696567f: Time spent bootstrapping tablet: real 0.181s user 0.031s sys 0.027s
I20250904 19:58:45.951527 16275 raft_consensus.cc:357] T 6b1b6476b0cc4a218f04c9e927199dc6 P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:45.952211 16275 raft_consensus.cc:383] T 6b1b6476b0cc4a218f04c9e927199dc6 P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:45.952495 16275 raft_consensus.cc:738] T 6b1b6476b0cc4a218f04c9e927199dc6 P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a083c1fe3c44686837e39e67696567f, State: Initialized, Role: FOLLOWER
I20250904 19:58:45.953236 16275 consensus_queue.cc:260] T 6b1b6476b0cc4a218f04c9e927199dc6 P 3a083c1fe3c44686837e39e67696567f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:45.957233 16279 tablet_bootstrap.cc:492] T 6b1b6476b0cc4a218f04c9e927199dc6 P 2a69af64672e4feb9c8d03702b501642: No bootstrap required, opened a new log
I20250904 19:58:45.958712 16279 ts_tablet_manager.cc:1397] T 6b1b6476b0cc4a218f04c9e927199dc6 P 2a69af64672e4feb9c8d03702b501642: Time spent bootstrapping tablet: real 0.075s user 0.015s sys 0.023s
I20250904 19:58:45.961344 16279 raft_consensus.cc:357] T 6b1b6476b0cc4a218f04c9e927199dc6 P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:45.962091 16279 raft_consensus.cc:383] T 6b1b6476b0cc4a218f04c9e927199dc6 P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:45.962424 16279 raft_consensus.cc:738] T 6b1b6476b0cc4a218f04c9e927199dc6 P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a69af64672e4feb9c8d03702b501642, State: Initialized, Role: FOLLOWER
I20250904 19:58:45.963249 16279 consensus_queue.cc:260] T 6b1b6476b0cc4a218f04c9e927199dc6 P 2a69af64672e4feb9c8d03702b501642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:45.971182 16013 catalog_manager.cc:1261] Loaded cluster ID: 69ebf69582804f788cbebb9f2b2f841d
I20250904 19:58:45.971491 16013 catalog_manager.cc:1554] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa: loading cluster ID for follower catalog manager: success
I20250904 19:58:45.972615 16281 tablet_bootstrap.cc:492] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca: No bootstrap required, opened a new log
I20250904 19:58:45.972800 16279 ts_tablet_manager.cc:1428] T 6b1b6476b0cc4a218f04c9e927199dc6 P 2a69af64672e4feb9c8d03702b501642: Time spent starting tablet: real 0.014s user 0.005s sys 0.007s
I20250904 19:58:45.973105 16281 ts_tablet_manager.cc:1397] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca: Time spent bootstrapping tablet: real 0.041s user 0.003s sys 0.006s
I20250904 19:58:45.974577 16012 catalog_manager.cc:1261] Loaded cluster ID: 69ebf69582804f788cbebb9f2b2f841d
I20250904 19:58:45.974848 16012 catalog_manager.cc:1554] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94: loading cluster ID for follower catalog manager: success
I20250904 19:58:45.975898 16281 raft_consensus.cc:357] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:45.976604 16281 raft_consensus.cc:383] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:45.976876 16281 raft_consensus.cc:738] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a51f25e351493b85dbd79ae7b2e3ca, State: Initialized, Role: FOLLOWER
I20250904 19:58:45.977638 16281 consensus_queue.cc:260] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:45.978379 16279 tablet_bootstrap.cc:492] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642: Bootstrap starting.
I20250904 19:58:45.984083 16279 tablet_bootstrap.cc:654] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:45.989897 16013 catalog_manager.cc:1576] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa: acquiring CA information for follower catalog manager: success
I20250904 19:58:45.995678 16275 ts_tablet_manager.cc:1428] T 6b1b6476b0cc4a218f04c9e927199dc6 P 3a083c1fe3c44686837e39e67696567f: Time spent starting tablet: real 0.047s user 0.007s sys 0.000s
I20250904 19:58:45.997340 16012 catalog_manager.cc:1576] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94: acquiring CA information for follower catalog manager: success
I20250904 19:58:45.999374 16013 catalog_manager.cc:1604] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:58:46.020212 16012 catalog_manager.cc:1604] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:58:46.036190 16281 ts_tablet_manager.cc:1428] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca: Time spent starting tablet: real 0.063s user 0.008s sys 0.005s
I20250904 19:58:46.041359 16281 tablet_bootstrap.cc:492] T 60b599c4af8b43a4b78b5721bf3b6dbd P 01a51f25e351493b85dbd79ae7b2e3ca: Bootstrap starting.
I20250904 19:58:46.042709 16277 raft_consensus.cc:491] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:46.043246 16277 raft_consensus.cc:513] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:46.045220 16277 leader_election.cc:290] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a69af64672e4feb9c8d03702b501642 (127.10.24.129:34527), 01a51f25e351493b85dbd79ae7b2e3ca (127.10.24.131:37413)
I20250904 19:58:46.046686 16229 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "60b599c4af8b43a4b78b5721bf3b6dbd" candidate_uuid: "3a083c1fe3c44686837e39e67696567f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" is_pre_election: true
I20250904 19:58:46.048059 16060 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "60b599c4af8b43a4b78b5721bf3b6dbd" candidate_uuid: "3a083c1fe3c44686837e39e67696567f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a69af64672e4feb9c8d03702b501642" is_pre_election: true
W20250904 19:58:46.048007 16110 leader_election.cc:343] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 01a51f25e351493b85dbd79ae7b2e3ca (127.10.24.131:37413): Illegal state: must be running to vote when last-logged opid is not known
I20250904 19:58:46.055652 16279 tablet_bootstrap.cc:492] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642: No bootstrap required, opened a new log
W20250904 19:58:46.056062 16108 leader_election.cc:343] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2a69af64672e4feb9c8d03702b501642 (127.10.24.129:34527): Illegal state: must be running to vote when last-logged opid is not known
I20250904 19:58:46.056188 16279 ts_tablet_manager.cc:1397] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642: Time spent bootstrapping tablet: real 0.078s user 0.026s sys 0.046s
I20250904 19:58:46.058492 16281 tablet_bootstrap.cc:654] T 60b599c4af8b43a4b78b5721bf3b6dbd P 01a51f25e351493b85dbd79ae7b2e3ca: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:46.058487 16108 leader_election.cc:304] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a083c1fe3c44686837e39e67696567f; no voters: 01a51f25e351493b85dbd79ae7b2e3ca, 2a69af64672e4feb9c8d03702b501642
I20250904 19:58:46.059072 16279 raft_consensus.cc:357] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:46.060585 16277 raft_consensus.cc:2747] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250904 19:58:46.060942 16279 raft_consensus.cc:383] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:46.064553 16279 raft_consensus.cc:738] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a69af64672e4feb9c8d03702b501642, State: Initialized, Role: FOLLOWER
I20250904 19:58:46.066323 16279 consensus_queue.cc:260] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:46.068642 16279 ts_tablet_manager.cc:1428] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642: Time spent starting tablet: real 0.012s user 0.003s sys 0.003s
I20250904 19:58:46.070379 16279 tablet_bootstrap.cc:492] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642: Bootstrap starting.
I20250904 19:58:46.078308 16279 tablet_bootstrap.cc:654] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:46.092917 16281 tablet_bootstrap.cc:492] T 60b599c4af8b43a4b78b5721bf3b6dbd P 01a51f25e351493b85dbd79ae7b2e3ca: No bootstrap required, opened a new log
I20250904 19:58:46.093441 16281 ts_tablet_manager.cc:1397] T 60b599c4af8b43a4b78b5721bf3b6dbd P 01a51f25e351493b85dbd79ae7b2e3ca: Time spent bootstrapping tablet: real 0.052s user 0.007s sys 0.004s
I20250904 19:58:46.095399 16279 tablet_bootstrap.cc:492] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642: No bootstrap required, opened a new log
I20250904 19:58:46.095914 16279 ts_tablet_manager.cc:1397] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642: Time spent bootstrapping tablet: real 0.026s user 0.018s sys 0.001s
I20250904 19:58:46.095695 16281 raft_consensus.cc:357] T 60b599c4af8b43a4b78b5721bf3b6dbd P 01a51f25e351493b85dbd79ae7b2e3ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:46.096369 16281 raft_consensus.cc:383] T 60b599c4af8b43a4b78b5721bf3b6dbd P 01a51f25e351493b85dbd79ae7b2e3ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:46.096676 16281 raft_consensus.cc:738] T 60b599c4af8b43a4b78b5721bf3b6dbd P 01a51f25e351493b85dbd79ae7b2e3ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a51f25e351493b85dbd79ae7b2e3ca, State: Initialized, Role: FOLLOWER
I20250904 19:58:46.097365 16281 consensus_queue.cc:260] T 60b599c4af8b43a4b78b5721bf3b6dbd P 01a51f25e351493b85dbd79ae7b2e3ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:46.098650 16279 raft_consensus.cc:357] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:46.099368 16279 raft_consensus.cc:383] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:46.099661 16279 raft_consensus.cc:738] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a69af64672e4feb9c8d03702b501642, State: Initialized, Role: FOLLOWER
I20250904 19:58:46.100375 16279 consensus_queue.cc:260] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:46.102591 16279 ts_tablet_manager.cc:1428] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20250904 19:58:46.103986 16281 ts_tablet_manager.cc:1428] T 60b599c4af8b43a4b78b5721bf3b6dbd P 01a51f25e351493b85dbd79ae7b2e3ca: Time spent starting tablet: real 0.010s user 0.008s sys 0.000s
I20250904 19:58:46.106793 16281 tablet_bootstrap.cc:492] T ae3869954dfa4f7db43da6d34dcefa6c P 01a51f25e351493b85dbd79ae7b2e3ca: Bootstrap starting.
I20250904 19:58:46.111986 16288 raft_consensus.cc:491] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:46.112468 16288 raft_consensus.cc:513] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:46.114308 16281 tablet_bootstrap.cc:654] T ae3869954dfa4f7db43da6d34dcefa6c P 01a51f25e351493b85dbd79ae7b2e3ca: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:46.114485 16288 leader_election.cc:290] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a083c1fe3c44686837e39e67696567f (127.10.24.130:43963), 2a69af64672e4feb9c8d03702b501642 (127.10.24.129:34527)
I20250904 19:58:46.159336 16281 tablet_bootstrap.cc:492] T ae3869954dfa4f7db43da6d34dcefa6c P 01a51f25e351493b85dbd79ae7b2e3ca: No bootstrap required, opened a new log
I20250904 19:58:46.160480 16281 ts_tablet_manager.cc:1397] T ae3869954dfa4f7db43da6d34dcefa6c P 01a51f25e351493b85dbd79ae7b2e3ca: Time spent bootstrapping tablet: real 0.054s user 0.025s sys 0.022s
I20250904 19:58:46.162108 16060 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6b1b6476b0cc4a218f04c9e927199dc6" candidate_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a69af64672e4feb9c8d03702b501642" is_pre_election: true
I20250904 19:58:46.162758 16060 raft_consensus.cc:2466] T 6b1b6476b0cc4a218f04c9e927199dc6 P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01a51f25e351493b85dbd79ae7b2e3ca in term 0.
I20250904 19:58:46.163218 16281 raft_consensus.cc:357] T ae3869954dfa4f7db43da6d34dcefa6c P 01a51f25e351493b85dbd79ae7b2e3ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:46.163990 16281 raft_consensus.cc:383] T ae3869954dfa4f7db43da6d34dcefa6c P 01a51f25e351493b85dbd79ae7b2e3ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:46.164328 16281 raft_consensus.cc:738] T ae3869954dfa4f7db43da6d34dcefa6c P 01a51f25e351493b85dbd79ae7b2e3ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a51f25e351493b85dbd79ae7b2e3ca, State: Initialized, Role: FOLLOWER
I20250904 19:58:46.164119 16151 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6b1b6476b0cc4a218f04c9e927199dc6" candidate_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a083c1fe3c44686837e39e67696567f" is_pre_election: true
I20250904 19:58:46.164594 16193 leader_election.cc:304] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a51f25e351493b85dbd79ae7b2e3ca, 2a69af64672e4feb9c8d03702b501642; no voters:
I20250904 19:58:46.165138 16151 raft_consensus.cc:2466] T 6b1b6476b0cc4a218f04c9e927199dc6 P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01a51f25e351493b85dbd79ae7b2e3ca in term 0.
I20250904 19:58:46.165484 16288 raft_consensus.cc:2802] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:46.165078 16281 consensus_queue.cc:260] T ae3869954dfa4f7db43da6d34dcefa6c P 01a51f25e351493b85dbd79ae7b2e3ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:46.165887 16288 raft_consensus.cc:491] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:46.166229 16288 raft_consensus.cc:3058] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:46.171048 16281 ts_tablet_manager.cc:1428] T ae3869954dfa4f7db43da6d34dcefa6c P 01a51f25e351493b85dbd79ae7b2e3ca: Time spent starting tablet: real 0.010s user 0.005s sys 0.002s
I20250904 19:58:46.173969 16288 raft_consensus.cc:513] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:46.175827 16288 leader_election.cc:290] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [CANDIDATE]: Term 1 election: Requested vote from peers 3a083c1fe3c44686837e39e67696567f (127.10.24.130:43963), 2a69af64672e4feb9c8d03702b501642 (127.10.24.129:34527)
I20250904 19:58:46.179118 16151 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6b1b6476b0cc4a218f04c9e927199dc6" candidate_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a083c1fe3c44686837e39e67696567f"
I20250904 19:58:46.179693 16151 raft_consensus.cc:3058] T 6b1b6476b0cc4a218f04c9e927199dc6 P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:46.183353 16060 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6b1b6476b0cc4a218f04c9e927199dc6" candidate_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a69af64672e4feb9c8d03702b501642"
I20250904 19:58:46.183925 16060 raft_consensus.cc:3058] T 6b1b6476b0cc4a218f04c9e927199dc6 P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:46.189507 16151 raft_consensus.cc:2466] T 6b1b6476b0cc4a218f04c9e927199dc6 P 3a083c1fe3c44686837e39e67696567f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a51f25e351493b85dbd79ae7b2e3ca in term 1.
I20250904 19:58:46.190939 16193 leader_election.cc:304] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a51f25e351493b85dbd79ae7b2e3ca, 3a083c1fe3c44686837e39e67696567f; no voters:
I20250904 19:58:46.191032 16060 raft_consensus.cc:2466] T 6b1b6476b0cc4a218f04c9e927199dc6 P 2a69af64672e4feb9c8d03702b501642 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a51f25e351493b85dbd79ae7b2e3ca in term 1.
I20250904 19:58:46.191831 16288 raft_consensus.cc:2802] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:46.192376 16288 raft_consensus.cc:695] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [term 1 LEADER]: Becoming Leader. State: Replica: 01a51f25e351493b85dbd79ae7b2e3ca, State: Running, Role: LEADER
I20250904 19:58:46.193181 16288 consensus_queue.cc:237] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:46.205173 15802 catalog_manager.cc:5582] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 01a51f25e351493b85dbd79ae7b2e3ca (127.10.24.131). New cstate: current_term: 1 leader_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } health_report { overall_health: HEALTHY } } }
I20250904 19:58:46.428756 16286 raft_consensus.cc:491] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:46.429306 16286 raft_consensus.cc:513] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:46.431708 16286 leader_election.cc:290] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a083c1fe3c44686837e39e67696567f (127.10.24.130:43963), 01a51f25e351493b85dbd79ae7b2e3ca (127.10.24.131:37413)
I20250904 19:58:46.446936 16229 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "ae3869954dfa4f7db43da6d34dcefa6c" candidate_uuid: "2a69af64672e4feb9c8d03702b501642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" is_pre_election: true
I20250904 19:58:46.447696 16229 raft_consensus.cc:2466] T ae3869954dfa4f7db43da6d34dcefa6c P 01a51f25e351493b85dbd79ae7b2e3ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a69af64672e4feb9c8d03702b501642 in term 0.
I20250904 19:58:46.450103 16025 leader_election.cc:304] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a51f25e351493b85dbd79ae7b2e3ca, 2a69af64672e4feb9c8d03702b501642; no voters:
I20250904 19:58:46.450239 16151 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "ae3869954dfa4f7db43da6d34dcefa6c" candidate_uuid: "2a69af64672e4feb9c8d03702b501642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a083c1fe3c44686837e39e67696567f" is_pre_election: true
I20250904 19:58:46.450839 16286 raft_consensus.cc:2802] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:46.450891 16151 raft_consensus.cc:2466] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a69af64672e4feb9c8d03702b501642 in term 0.
I20250904 19:58:46.451198 16286 raft_consensus.cc:491] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:46.451601 16286 raft_consensus.cc:3058] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:46.463204 16286 raft_consensus.cc:513] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:46.464985 16286 leader_election.cc:290] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [CANDIDATE]: Term 1 election: Requested vote from peers 3a083c1fe3c44686837e39e67696567f (127.10.24.130:43963), 01a51f25e351493b85dbd79ae7b2e3ca (127.10.24.131:37413)
I20250904 19:58:46.465582 16151 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "ae3869954dfa4f7db43da6d34dcefa6c" candidate_uuid: "2a69af64672e4feb9c8d03702b501642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a083c1fe3c44686837e39e67696567f"
I20250904 19:58:46.465860 16229 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "ae3869954dfa4f7db43da6d34dcefa6c" candidate_uuid: "2a69af64672e4feb9c8d03702b501642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a51f25e351493b85dbd79ae7b2e3ca"
I20250904 19:58:46.466161 16151 raft_consensus.cc:3058] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:46.466393 16229 raft_consensus.cc:3058] T ae3869954dfa4f7db43da6d34dcefa6c P 01a51f25e351493b85dbd79ae7b2e3ca [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:46.478405 16229 raft_consensus.cc:2466] T ae3869954dfa4f7db43da6d34dcefa6c P 01a51f25e351493b85dbd79ae7b2e3ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a69af64672e4feb9c8d03702b501642 in term 1.
I20250904 19:58:46.478705 16151 raft_consensus.cc:2466] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a69af64672e4feb9c8d03702b501642 in term 1.
I20250904 19:58:46.480072 16024 leader_election.cc:304] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a69af64672e4feb9c8d03702b501642, 3a083c1fe3c44686837e39e67696567f; no voters:
I20250904 19:58:46.481829 16286 raft_consensus.cc:2802] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:46.485364 16286 raft_consensus.cc:695] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [term 1 LEADER]: Becoming Leader. State: Replica: 2a69af64672e4feb9c8d03702b501642, State: Running, Role: LEADER
I20250904 19:58:46.487004 16307 raft_consensus.cc:491] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:46.487591 16307 raft_consensus.cc:513] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:46.490061 16307 leader_election.cc:290] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a083c1fe3c44686837e39e67696567f (127.10.24.130:43963), 01a51f25e351493b85dbd79ae7b2e3ca (127.10.24.131:37413)
I20250904 19:58:46.490307 16151 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "60b599c4af8b43a4b78b5721bf3b6dbd" candidate_uuid: "2a69af64672e4feb9c8d03702b501642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a083c1fe3c44686837e39e67696567f" is_pre_election: true
I20250904 19:58:46.491159 16151 raft_consensus.cc:2466] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a69af64672e4feb9c8d03702b501642 in term 0.
I20250904 19:58:46.492084 16024 leader_election.cc:304] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a69af64672e4feb9c8d03702b501642, 3a083c1fe3c44686837e39e67696567f; no voters:
I20250904 19:58:46.493168 16286 consensus_queue.cc:237] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:46.494939 16307 raft_consensus.cc:2802] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:46.495368 16307 raft_consensus.cc:491] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:46.495769 16307 raft_consensus.cc:3058] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:46.497423 16229 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "60b599c4af8b43a4b78b5721bf3b6dbd" candidate_uuid: "2a69af64672e4feb9c8d03702b501642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" is_pre_election: true
I20250904 19:58:46.497900 16229 raft_consensus.cc:2466] T 60b599c4af8b43a4b78b5721bf3b6dbd P 01a51f25e351493b85dbd79ae7b2e3ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a69af64672e4feb9c8d03702b501642 in term 0.
I20250904 19:58:46.506624 15802 catalog_manager.cc:5582] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a69af64672e4feb9c8d03702b501642 (127.10.24.129). New cstate: current_term: 1 leader_uuid: "2a69af64672e4feb9c8d03702b501642" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } health_report { overall_health: UNKNOWN } } }
I20250904 19:58:46.508920 16307 raft_consensus.cc:513] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:46.513562 16307 leader_election.cc:290] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [CANDIDATE]: Term 1 election: Requested vote from peers 3a083c1fe3c44686837e39e67696567f (127.10.24.130:43963), 01a51f25e351493b85dbd79ae7b2e3ca (127.10.24.131:37413)
I20250904 19:58:46.514096 16151 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "60b599c4af8b43a4b78b5721bf3b6dbd" candidate_uuid: "2a69af64672e4feb9c8d03702b501642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a083c1fe3c44686837e39e67696567f"
I20250904 19:58:46.514680 16151 raft_consensus.cc:3058] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:46.519013 16229 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "60b599c4af8b43a4b78b5721bf3b6dbd" candidate_uuid: "2a69af64672e4feb9c8d03702b501642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a51f25e351493b85dbd79ae7b2e3ca"
I20250904 19:58:46.519599 16229 raft_consensus.cc:3058] T 60b599c4af8b43a4b78b5721bf3b6dbd P 01a51f25e351493b85dbd79ae7b2e3ca [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:46.526343 16229 raft_consensus.cc:2466] T 60b599c4af8b43a4b78b5721bf3b6dbd P 01a51f25e351493b85dbd79ae7b2e3ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a69af64672e4feb9c8d03702b501642 in term 1.
I20250904 19:58:46.528853 16151 raft_consensus.cc:2466] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a69af64672e4feb9c8d03702b501642 in term 1.
I20250904 19:58:46.529553 16025 leader_election.cc:304] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a51f25e351493b85dbd79ae7b2e3ca, 2a69af64672e4feb9c8d03702b501642; no voters:
I20250904 19:58:46.530357 16307 raft_consensus.cc:2802] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:46.530869 16307 raft_consensus.cc:695] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [term 1 LEADER]: Becoming Leader. State: Replica: 2a69af64672e4feb9c8d03702b501642, State: Running, Role: LEADER
I20250904 19:58:46.531735 16307 consensus_queue.cc:237] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } }
I20250904 19:58:46.546859 15802 catalog_manager.cc:5582] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a69af64672e4feb9c8d03702b501642 (127.10.24.129). New cstate: current_term: 1 leader_uuid: "2a69af64672e4feb9c8d03702b501642" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 } health_report { overall_health: UNKNOWN } } }
I20250904 19:58:46.603070 16288 consensus_queue.cc:1035] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:46.629534 16314 consensus_queue.cc:1035] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a69af64672e4feb9c8d03702b501642" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250904 19:58:47.037223 16286 consensus_queue.cc:1035] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:47.043421 16286 consensus_queue.cc:1035] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:47.101791 16286 consensus_queue.cc:1035] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a51f25e351493b85dbd79ae7b2e3ca" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 37413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:47.107703 16286 consensus_queue.cc:1035] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a083c1fe3c44686837e39e67696567f" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 43963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:47.198817 10338 master.cc:561] Master@127.10.24.190:33243 shutting down...
I20250904 19:58:47.233757 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:47.234542 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:47.234885 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 8f10ffaee87a4bd8b48b60ca5adad90c: stopping tablet replica
I20250904 19:58:47.711201 10338 master.cc:583] Master@127.10.24.190:33243 shutdown complete.
I20250904 19:58:48.551384 16362 raft_consensus.cc:491] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f10ffaee87a4bd8b48b60ca5adad90c)
I20250904 19:58:48.551928 16362 raft_consensus.cc:513] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } }
I20250904 19:58:48.553649 16362 leader_election.cc:290] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f10ffaee87a4bd8b48b60ca5adad90c (127.10.24.190:33243), d5ee58d0539642938a4acad2774191aa (127.10.24.188:39667)
I20250904 19:58:48.554541 15945 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e1e217baccf24181889482aa9a783e94" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "d5ee58d0539642938a4acad2774191aa" is_pre_election: true
W20250904 19:58:48.559345 15852 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.24.190:33243: connect: Connection refused (error 111) [suppressed 12 similar messages]
W20250904 19:58:48.562378 15852 leader_election.cc:336] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8f10ffaee87a4bd8b48b60ca5adad90c (127.10.24.190:33243): Network error: Client connection negotiation failed: client connection to 127.10.24.190:33243: connect: Connection refused (error 111)
I20250904 19:58:48.562736 15852 leader_election.cc:304] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e1e217baccf24181889482aa9a783e94; no voters: 8f10ffaee87a4bd8b48b60ca5adad90c, d5ee58d0539642938a4acad2774191aa
I20250904 19:58:48.563318 16362 raft_consensus.cc:2747] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 19:58:48.649529 16365 raft_consensus.cc:491] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8f10ffaee87a4bd8b48b60ca5adad90c)
I20250904 19:58:48.650182 16365 raft_consensus.cc:513] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } }
I20250904 19:58:48.654704 15879 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d5ee58d0539642938a4acad2774191aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "e1e217baccf24181889482aa9a783e94" is_pre_election: true
I20250904 19:58:48.655401 15879 raft_consensus.cc:2466] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5ee58d0539642938a4acad2774191aa in term 1.
I20250904 19:58:48.656769 15920 leader_election.cc:304] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5ee58d0539642938a4acad2774191aa, e1e217baccf24181889482aa9a783e94; no voters:
I20250904 19:58:48.658396 16367 raft_consensus.cc:2802] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250904 19:58:48.658800 16365 leader_election.cc:290] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8f10ffaee87a4bd8b48b60ca5adad90c (127.10.24.190:33243), e1e217baccf24181889482aa9a783e94 (127.10.24.189:38645)
I20250904 19:58:48.660338 16367 raft_consensus.cc:491] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8f10ffaee87a4bd8b48b60ca5adad90c)
I20250904 19:58:48.660770 16367 raft_consensus.cc:3058] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [term 1 FOLLOWER]: Advancing to term 2
W20250904 19:58:48.661720 15918 leader_election.cc:336] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8f10ffaee87a4bd8b48b60ca5adad90c (127.10.24.190:33243): Network error: Client connection negotiation failed: client connection to 127.10.24.190:33243: connect: Connection refused (error 111)
I20250904 19:58:48.669431 16367 raft_consensus.cc:513] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } }
I20250904 19:58:48.672283 15879 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d5ee58d0539642938a4acad2774191aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "e1e217baccf24181889482aa9a783e94"
I20250904 19:58:48.672859 15879 raft_consensus.cc:3058] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [term 1 FOLLOWER]: Advancing to term 2
W20250904 19:58:48.677614 15918 leader_election.cc:336] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8f10ffaee87a4bd8b48b60ca5adad90c (127.10.24.190:33243): Network error: Client connection negotiation failed: client connection to 127.10.24.190:33243: connect: Connection refused (error 111)
I20250904 19:58:48.684602 16367 leader_election.cc:290] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [CANDIDATE]: Term 2 election: Requested vote from peers 8f10ffaee87a4bd8b48b60ca5adad90c (127.10.24.190:33243), e1e217baccf24181889482aa9a783e94 (127.10.24.189:38645)
I20250904 19:58:48.685721 15879 raft_consensus.cc:2466] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5ee58d0539642938a4acad2774191aa in term 2.
I20250904 19:58:48.687253 15920 leader_election.cc:304] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d5ee58d0539642938a4acad2774191aa, e1e217baccf24181889482aa9a783e94; no voters: 8f10ffaee87a4bd8b48b60ca5adad90c
I20250904 19:58:48.688530 16367 raft_consensus.cc:2802] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [term 2 FOLLOWER]: Leader election won for term 2
I20250904 19:58:48.689221 16367 raft_consensus.cc:695] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [term 2 LEADER]: Becoming Leader. State: Replica: d5ee58d0539642938a4acad2774191aa, State: Running, Role: LEADER
I20250904 19:58:48.690071 16367 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } }
I20250904 19:58:48.696084 16365 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5ee58d0539642938a4acad2774191aa. Latest consensus state: current_term: 2 leader_uuid: "d5ee58d0539642938a4acad2774191aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } } }
I20250904 19:58:48.696705 16365 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:48.698348 16370 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 19:58:48.707064 16370 catalog_manager.cc:671] Loaded metadata for table ksck-test-table [id=8a653c26d4bc4f7ebef7bb6fd3623fb9]
I20250904 19:58:48.713371 16370 tablet_loader.cc:96] loaded metadata for tablet 60b599c4af8b43a4b78b5721bf3b6dbd (table ksck-test-table [id=8a653c26d4bc4f7ebef7bb6fd3623fb9])
I20250904 19:58:48.714738 16370 tablet_loader.cc:96] loaded metadata for tablet 6b1b6476b0cc4a218f04c9e927199dc6 (table ksck-test-table [id=8a653c26d4bc4f7ebef7bb6fd3623fb9])
I20250904 19:58:48.715801 16370 tablet_loader.cc:96] loaded metadata for tablet ae3869954dfa4f7db43da6d34dcefa6c (table ksck-test-table [id=8a653c26d4bc4f7ebef7bb6fd3623fb9])
I20250904 19:58:48.716933 16370 catalog_manager.cc:1486] Initializing Kudu cluster ID...
W20250904 19:58:48.717463 16250 debug-util.cc:398] Leaking SignalData structure 0x7b08001e6900 after lost signal to thread 10341
W20250904 19:58:48.720533 16250 debug-util.cc:398] Leaking SignalData structure 0x7b0800231ce0 after lost signal to thread 16084
W20250904 19:58:48.724589 16250 debug-util.cc:398] Leaking SignalData structure 0x7b08001259e0 after lost signal to thread 16175
W20250904 19:58:48.725627 16250 debug-util.cc:398] Leaking SignalData structure 0x7b080025b3a0 after lost signal to thread 16253
I20250904 19:58:48.721136 16370 catalog_manager.cc:1261] Loaded cluster ID: 69ebf69582804f788cbebb9f2b2f841d
I20250904 19:58:48.731010 16370 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 19:58:48.735761 16370 catalog_manager.cc:1506] Loading token signing keys...
I20250904 19:58:48.739881 16370 catalog_manager.cc:5966] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa: Loaded TSK: 0
I20250904 19:58:48.741233 16370 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 19:58:48.760385 16255 heartbeater.cc:499] Master 127.10.24.188:39667 was elected leader, sending a full tablet report...
I20250904 19:58:48.760499 16177 heartbeater.cc:499] Master 127.10.24.188:39667 was elected leader, sending a full tablet report...
I20250904 19:58:48.763374 16086 heartbeater.cc:499] Master 127.10.24.188:39667 was elected leader, sending a full tablet report...
I20250904 19:58:49.025256 16218 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250904 19:58:49.031792 16141 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
I20250904 19:58:49.050573 16049 tablet_service.cc:1430] Tablet server has 2 leaders and 0 scanners
I20250904 19:58:49.110708 15879 raft_consensus.cc:1273] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [term 2 FOLLOWER]: Refusing update from remote peer d5ee58d0539642938a4acad2774191aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 9. Preceding OpId from leader: term: 2 index: 10. (index mismatch)
I20250904 19:58:49.112614 16365 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20250904 19:58:49.168411 16381 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5ee58d0539642938a4acad2774191aa. Latest consensus state: current_term: 2 leader_uuid: "d5ee58d0539642938a4acad2774191aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } } }
I20250904 19:58:49.169164 16381 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:49.178602 16367 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "d5ee58d0539642938a4acad2774191aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } } }
I20250904 19:58:49.179407 16367 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:49.196334 16381 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "d5ee58d0539642938a4acad2774191aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8f10ffaee87a4bd8b48b60ca5adad90c" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 33243 } } peers { permanent_uuid: "e1e217baccf24181889482aa9a783e94" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 38645 } } peers { permanent_uuid: "d5ee58d0539642938a4acad2774191aa" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 39667 } } }
I20250904 19:58:49.197103 16381 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [sys.catalog]: This master's current role is: FOLLOWER
W20250904 19:58:49.237826 15918 consensus_peers.cc:489] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa -> Peer 8f10ffaee87a4bd8b48b60ca5adad90c (127.10.24.190:33243): Couldn't send request to peer 8f10ffaee87a4bd8b48b60ca5adad90c. Status: Network error: Client connection negotiation failed: client connection to 127.10.24.190:33243: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 19:58:49.515820 10338 tablet_server.cc:178] TabletServer@127.10.24.129:0 shutting down...
I20250904 19:58:49.578126 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:49.578835 10338 tablet_replica.cc:331] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642: stopping tablet replica
I20250904 19:58:49.579694 10338 raft_consensus.cc:2241] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:49.580772 10338 raft_consensus.cc:2270] T ae3869954dfa4f7db43da6d34dcefa6c P 2a69af64672e4feb9c8d03702b501642 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:49.587090 10338 tablet_replica.cc:331] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642: stopping tablet replica
I20250904 19:58:49.587836 10338 raft_consensus.cc:2241] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:49.588872 10338 raft_consensus.cc:2270] T 60b599c4af8b43a4b78b5721bf3b6dbd P 2a69af64672e4feb9c8d03702b501642 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:49.591375 10338 tablet_replica.cc:331] T 6b1b6476b0cc4a218f04c9e927199dc6 P 2a69af64672e4feb9c8d03702b501642: stopping tablet replica
I20250904 19:58:49.592013 10338 raft_consensus.cc:2241] T 6b1b6476b0cc4a218f04c9e927199dc6 P 2a69af64672e4feb9c8d03702b501642 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:49.592514 10338 raft_consensus.cc:2270] T 6b1b6476b0cc4a218f04c9e927199dc6 P 2a69af64672e4feb9c8d03702b501642 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:49.620375 10338 tablet_server.cc:195] TabletServer@127.10.24.129:0 shutdown complete.
I20250904 19:58:49.637475 10338 tablet_server.cc:178] TabletServer@127.10.24.130:0 shutting down...
I20250904 19:58:49.700886 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:49.701567 10338 tablet_replica.cc:331] T 6b1b6476b0cc4a218f04c9e927199dc6 P 3a083c1fe3c44686837e39e67696567f: stopping tablet replica
I20250904 19:58:49.702359 10338 raft_consensus.cc:2241] T 6b1b6476b0cc4a218f04c9e927199dc6 P 3a083c1fe3c44686837e39e67696567f [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:49.702924 10338 raft_consensus.cc:2270] T 6b1b6476b0cc4a218f04c9e927199dc6 P 3a083c1fe3c44686837e39e67696567f [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:49.705436 10338 tablet_replica.cc:331] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f: stopping tablet replica
I20250904 19:58:49.706126 10338 raft_consensus.cc:2241] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:49.706681 10338 raft_consensus.cc:2270] T 60b599c4af8b43a4b78b5721bf3b6dbd P 3a083c1fe3c44686837e39e67696567f [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:49.709088 10338 tablet_replica.cc:331] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f: stopping tablet replica
I20250904 19:58:49.709743 10338 raft_consensus.cc:2241] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:49.710234 10338 raft_consensus.cc:2270] T ae3869954dfa4f7db43da6d34dcefa6c P 3a083c1fe3c44686837e39e67696567f [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:49.733395 10338 tablet_server.cc:195] TabletServer@127.10.24.130:0 shutdown complete.
I20250904 19:58:49.750180 10338 tablet_server.cc:178] TabletServer@127.10.24.131:0 shutting down...
I20250904 19:58:49.789283 10338 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 19:58:49.789968 10338 tablet_replica.cc:331] T ae3869954dfa4f7db43da6d34dcefa6c P 01a51f25e351493b85dbd79ae7b2e3ca: stopping tablet replica
I20250904 19:58:49.790714 10338 raft_consensus.cc:2241] T ae3869954dfa4f7db43da6d34dcefa6c P 01a51f25e351493b85dbd79ae7b2e3ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:49.791263 10338 raft_consensus.cc:2270] T ae3869954dfa4f7db43da6d34dcefa6c P 01a51f25e351493b85dbd79ae7b2e3ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:49.793766 10338 tablet_replica.cc:331] T 60b599c4af8b43a4b78b5721bf3b6dbd P 01a51f25e351493b85dbd79ae7b2e3ca: stopping tablet replica
I20250904 19:58:49.794463 10338 raft_consensus.cc:2241] T 60b599c4af8b43a4b78b5721bf3b6dbd P 01a51f25e351493b85dbd79ae7b2e3ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:49.794971 10338 raft_consensus.cc:2270] T 60b599c4af8b43a4b78b5721bf3b6dbd P 01a51f25e351493b85dbd79ae7b2e3ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:49.797415 10338 tablet_replica.cc:331] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca: stopping tablet replica
I20250904 19:58:49.798080 10338 raft_consensus.cc:2241] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:49.799060 10338 raft_consensus.cc:2270] T 6b1b6476b0cc4a218f04c9e927199dc6 P 01a51f25e351493b85dbd79ae7b2e3ca [term 1 FOLLOWER]: Raft consensus is shut down!
W20250904 19:58:50.820655 10338 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 16250)
W20250904 19:58:51.673542 15918 consensus_peers.cc:489] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa -> Peer 8f10ffaee87a4bd8b48b60ca5adad90c (127.10.24.190:33243): Couldn't send request to peer 8f10ffaee87a4bd8b48b60ca5adad90c. Status: Network error: Client connection negotiation failed: client connection to 127.10.24.190:33243: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20250904 19:58:51.821231 10338 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 16250)
I20250904 19:58:52.423646 10338 tablet_server.cc:195] TabletServer@127.10.24.131:0 shutdown complete.
I20250904 19:58:52.447479 10338 master.cc:561] Master@127.10.24.189:38645 shutting down...
I20250904 19:58:52.472606 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:52.473407 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:52.473726 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P e1e217baccf24181889482aa9a783e94: stopping tablet replica
W20250904 19:58:52.486785 15920 consensus_peers.cc:489] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa -> Peer e1e217baccf24181889482aa9a783e94 (127.10.24.189:38645): Couldn't send request to peer e1e217baccf24181889482aa9a783e94. Status: Network error: Client connection negotiation failed: client connection to 127.10.24.189:38645: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 19:58:52.494992 10338 master.cc:583] Master@127.10.24.189:38645 shutdown complete.
I20250904 19:58:52.515077 10338 master.cc:561] Master@127.10.24.188:39667 shutting down...
I20250904 19:58:52.536559 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [term 2 LEADER]: Raft consensus shutting down.
I20250904 19:58:52.537696 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:52.538058 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P d5ee58d0539642938a4acad2774191aa: stopping tablet replica
I20250904 19:58:52.560019 10338 master.cc:583] Master@127.10.24.188:39667 shutdown complete.
[ OK ] RemoteKsckTest.TestLeaderMasterDown (7988 ms)
[ RUN ] RemoteKsckTest.TestClusterWithLocation
I20250904 19:58:52.609778 10338 test_util.cc:276] Using random seed: 646391002
I20250904 19:58:52.610818 10338 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.24.190:43389,127.10.24.189:41319,127.10.24.188:37777
I20250904 19:58:52.612263 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:52.618259 16393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:52.618332 16392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:52.619043 16395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:52.620745 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:52.621582 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:52.621783 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:52.621923 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015932621906 us; error 0 us; skew 500 ppm
I20250904 19:58:52.622409 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:52.624845 10338 webserver.cc:480] Webserver started at http://127.10.24.190:41449/ using document root <none> and password file <none>
I20250904 19:58:52.625319 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:52.625490 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:52.625726 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:52.626981 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-0-root/instance:
uuid: "de9343d4ee8749d5b98effa595ef18e3"
format_stamp: "Formatted at 2025-09-04 19:58:52 on dist-test-slave-f3xj"
I20250904 19:58:52.632045 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20250904 19:58:52.635879 16400 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:52.637485 10338 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20250904 19:58:52.637926 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-0-root
uuid: "de9343d4ee8749d5b98effa595ef18e3"
format_stamp: "Formatted at 2025-09-04 19:58:52 on dist-test-slave-f3xj"
I20250904 19:58:52.638535 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:52.653378 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:52.654690 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:52.694701 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.190:43389
I20250904 19:58:52.694802 16451 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.190:43389 every 8 connection(s)
I20250904 19:58:52.698731 16452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:52.698840 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:52.704612 16454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:52.705909 16455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:52.708469 10338 server_base.cc:1047] running on GCE node
W20250904 19:58:52.708602 16457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:52.710386 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:52.710598 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:52.710749 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015932710732 us; error 0 us; skew 500 ppm
I20250904 19:58:52.711293 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:52.711303 16452 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:52.714895 10338 webserver.cc:480] Webserver started at http://127.10.24.189:46881/ using document root <none> and password file <none>
I20250904 19:58:52.715467 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:52.715727 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:52.715987 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:52.717329 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-1-root/instance:
uuid: "14c649e56e32428798b4895763179e23"
format_stamp: "Formatted at 2025-09-04 19:58:52 on dist-test-slave-f3xj"
I20250904 19:58:52.722678 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20250904 19:58:52.726504 16464 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:52.727399 10338 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20250904 19:58:52.727766 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-1-root
uuid: "14c649e56e32428798b4895763179e23"
format_stamp: "Formatted at 2025-09-04 19:58:52 on dist-test-slave-f3xj"
I20250904 19:58:52.728122 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:52.730911 16452 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:58:52.737169 16452 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.189:41319: Network error: Client connection negotiation failed: client connection to 127.10.24.189:41319: connect: Connection refused (error 111)
I20250904 19:58:52.759495 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:52.760838 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:52.780664 16452 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } attempt: 1
W20250904 19:58:52.785902 16452 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.189:41319: Network error: Client connection negotiation failed: client connection to 127.10.24.189:41319: connect: Connection refused (error 111)
I20250904 19:58:52.801959 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.189:41319
I20250904 19:58:52.802083 16516 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.189:41319 every 8 connection(s)
I20250904 19:58:52.806078 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:52.806293 16517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
W20250904 19:58:52.812376 16519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:52.812809 16520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:52.817883 16517 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:52.821631 10338 server_base.cc:1047] running on GCE node
W20250904 19:58:52.822156 16522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:52.823082 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:52.823300 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:52.823457 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015932823440 us; error 0 us; skew 500 ppm
I20250904 19:58:52.823982 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:52.826782 10338 webserver.cc:480] Webserver started at http://127.10.24.188:38523/ using document root <none> and password file <none>
I20250904 19:58:52.827448 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:52.827687 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:52.828011 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:52.829497 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-2-root/instance:
uuid: "3609a8f04c864dbeb9e91fff62ecc4ca"
format_stamp: "Formatted at 2025-09-04 19:58:52 on dist-test-slave-f3xj"
I20250904 19:58:52.831143 16517 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:52.835307 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20250904 19:58:52.839870 16529 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:52.840855 10338 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20250904 19:58:52.841229 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-2-root
uuid: "3609a8f04c864dbeb9e91fff62ecc4ca"
format_stamp: "Formatted at 2025-09-04 19:58:52 on dist-test-slave-f3xj"
I20250904 19:58:52.841528 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:52.843824 16517 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:58:52.849205 16517 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:37777: Network error: Client connection negotiation failed: client connection to 127.10.24.188:37777: connect: Connection refused (error 111)
I20250904 19:58:52.855419 16452 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } attempt: 2
I20250904 19:58:52.866127 16452 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } has no permanent_uuid. Determining permanent_uuid...
W20250904 19:58:52.870615 16452 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:37777: Network error: Client connection negotiation failed: client connection to 127.10.24.188:37777: connect: Connection refused (error 111)
I20250904 19:58:52.871814 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:52.873052 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:52.896756 16517 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } attempt: 1
W20250904 19:58:52.901094 16517 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:37777: Network error: Client connection negotiation failed: client connection to 127.10.24.188:37777: connect: Connection refused (error 111)
I20250904 19:58:52.909161 16452 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } attempt: 1
W20250904 19:58:52.913938 16452 consensus_peers.cc:648] Error getting permanent uuid from config peer 127.10.24.188:37777: Network error: Client connection negotiation failed: client connection to 127.10.24.188:37777: connect: Connection refused (error 111)
I20250904 19:58:52.915172 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.188:37777
I20250904 19:58:52.915297 16581 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.188:37777 every 8 connection(s)
I20250904 19:58:52.918232 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 19:58:52.919025 16582 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:52.926467 16582 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:52.937278 16582 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:52.946421 16582 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } has no permanent_uuid. Determining permanent_uuid...
I20250904 19:58:52.962997 16582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca: Bootstrap starting.
I20250904 19:58:52.964661 16517 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } attempt: 2
I20250904 19:58:52.968469 16582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:52.973263 16582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca: No bootstrap required, opened a new log
I20250904 19:58:52.976130 16582 raft_consensus.cc:357] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } }
I20250904 19:58:52.976682 16582 raft_consensus.cc:383] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:52.976912 16582 raft_consensus.cc:738] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3609a8f04c864dbeb9e91fff62ecc4ca, State: Initialized, Role: FOLLOWER
I20250904 19:58:52.977499 16582 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } }
I20250904 19:58:52.979506 16587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } } }
I20250904 19:58:52.980340 16587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:52.980702 16582 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:52.982154 16517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23: Bootstrap starting.
I20250904 19:58:52.983458 16452 consensus_peers.cc:658] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } attempt: 2
I20250904 19:58:52.990267 16517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:52.996312 16517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23: No bootstrap required, opened a new log
I20250904 19:58:52.998675 16517 raft_consensus.cc:357] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } }
I20250904 19:58:52.999257 16517 raft_consensus.cc:383] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:52.999511 16517 raft_consensus.cc:738] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14c649e56e32428798b4895763179e23, State: Initialized, Role: FOLLOWER
I20250904 19:58:53.000191 16517 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } }
W20250904 19:58:53.000862 16599 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:58:53.001241 16599 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250904 19:58:53.002460 16600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } } }
I20250904 19:58:53.003257 16600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:53.004052 16517 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:53.004501 16452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3: Bootstrap starting.
I20250904 19:58:53.009889 16452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:53.014904 16452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3: No bootstrap required, opened a new log
I20250904 19:58:53.017215 16452 raft_consensus.cc:357] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } }
I20250904 19:58:53.017784 16452 raft_consensus.cc:383] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:53.018013 16452 raft_consensus.cc:738] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de9343d4ee8749d5b98effa595ef18e3, State: Initialized, Role: FOLLOWER
I20250904 19:58:53.018617 16452 consensus_queue.cc:260] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } }
W20250904 19:58:53.020066 16612 catalog_manager.cc:1560] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:58:53.020387 16612 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20250904 19:58:53.020646 16613 sys_catalog.cc:455] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } } }
I20250904 19:58:53.021442 16613 sys_catalog.cc:458] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:53.022203 16452 sys_catalog.cc:564] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:53.032774 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 1
I20250904 19:58:53.033030 10338 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 2
I20250904 19:58:53.034508 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:53.035727 16624 catalog_manager.cc:1560] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20250904 19:58:53.036016 16624 catalog_manager.cc:875] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20250904 19:58:53.041556 16625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:53.042898 16626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:53.043776 16628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:53.044842 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:53.045743 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:53.045935 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:53.046101 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015933046086 us; error 0 us; skew 500 ppm
I20250904 19:58:53.046646 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:53.049026 10338 webserver.cc:480] Webserver started at http://127.10.24.129:42001/ using document root <none> and password file <none>
I20250904 19:58:53.049568 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:53.049759 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:53.049999 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:53.051126 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/ts-0-root/instance:
uuid: "34fcdc21eb7e477ebab2618592e9c276"
format_stamp: "Formatted at 2025-09-04 19:58:53 on dist-test-slave-f3xj"
I20250904 19:58:53.055549 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20250904 19:58:53.058928 16633 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:53.059692 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250904 19:58:53.059983 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/ts-0-root
uuid: "34fcdc21eb7e477ebab2618592e9c276"
format_stamp: "Formatted at 2025-09-04 19:58:53 on dist-test-slave-f3xj"
I20250904 19:58:53.060246 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:53.069118 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:53.070299 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:53.071797 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:53.074062 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:53.074259 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:53.074499 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:53.074664 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:53.114514 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.129:46843
I20250904 19:58:53.114629 16695 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.129:46843 every 8 connection(s)
I20250904 19:58:53.123240 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:53.133692 16703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:53.135816 16696 heartbeater.cc:344] Connected to a master server at 127.10.24.188:37777
I20250904 19:58:53.136286 16696 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:53.137197 16696 heartbeater.cc:507] Master 127.10.24.188:37777 requested a full tablet report, sending...
I20250904 19:58:53.139474 16697 heartbeater.cc:344] Connected to a master server at 127.10.24.189:41319
I20250904 19:58:53.139904 16697 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:53.140475 16547 ts_manager.cc:194] Registered new tserver with Master: 34fcdc21eb7e477ebab2618592e9c276 (127.10.24.129:46843)
I20250904 19:58:53.140822 16697 heartbeater.cc:507] Master 127.10.24.189:41319 requested a full tablet report, sending...
I20250904 19:58:53.145687 16482 ts_manager.cc:194] Registered new tserver with Master: 34fcdc21eb7e477ebab2618592e9c276 (127.10.24.129:46843)
I20250904 19:58:53.147388 16698 heartbeater.cc:344] Connected to a master server at 127.10.24.190:43389
I20250904 19:58:53.147898 16698 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:53.148823 16698 heartbeater.cc:507] Master 127.10.24.190:43389 requested a full tablet report, sending...
W20250904 19:58:53.149307 16704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:53.151224 16706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:53.151901 16417 ts_manager.cc:194] Registered new tserver with Master: 34fcdc21eb7e477ebab2618592e9c276 (127.10.24.129:46843)
I20250904 19:58:53.154553 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:53.156641 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:53.156847 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:53.156988 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015933156976 us; error 0 us; skew 500 ppm
I20250904 19:58:53.157554 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:53.160171 10338 webserver.cc:480] Webserver started at http://127.10.24.130:32967/ using document root <none> and password file <none>
I20250904 19:58:53.160745 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:53.160948 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:53.161197 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:53.162319 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/ts-1-root/instance:
uuid: "1121c8a9c5f64923856f52dd7c39b015"
format_stamp: "Formatted at 2025-09-04 19:58:53 on dist-test-slave-f3xj"
I20250904 19:58:53.167764 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20250904 19:58:53.171196 16711 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:53.171957 10338 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250904 19:58:53.172346 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/ts-1-root
uuid: "1121c8a9c5f64923856f52dd7c39b015"
format_stamp: "Formatted at 2025-09-04 19:58:53 on dist-test-slave-f3xj"
I20250904 19:58:53.172654 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:53.176265 16613 raft_consensus.cc:491] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:53.176898 16613 raft_consensus.cc:513] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } }
I20250904 19:58:53.178890 16613 leader_election.cc:290] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14c649e56e32428798b4895763179e23 (127.10.24.189:41319), 3609a8f04c864dbeb9e91fff62ecc4ca (127.10.24.188:37777)
I20250904 19:58:53.179701 16492 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de9343d4ee8749d5b98effa595ef18e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14c649e56e32428798b4895763179e23" is_pre_election: true
I20250904 19:58:53.180419 16557 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de9343d4ee8749d5b98effa595ef18e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" is_pre_election: true
I20250904 19:58:53.181316 16557 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de9343d4ee8749d5b98effa595ef18e3 in term 0.
I20250904 19:58:53.181283 16492 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate de9343d4ee8749d5b98effa595ef18e3 in term 0.
I20250904 19:58:53.182747 16401 leader_election.cc:304] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3609a8f04c864dbeb9e91fff62ecc4ca, de9343d4ee8749d5b98effa595ef18e3; no voters:
I20250904 19:58:53.183682 16613 raft_consensus.cc:2802] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:53.183992 16613 raft_consensus.cc:491] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:53.184279 16613 raft_consensus.cc:3058] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:53.190461 16613 raft_consensus.cc:513] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } }
I20250904 19:58:53.191815 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:53.192020 16613 leader_election.cc:290] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [CANDIDATE]: Term 1 election: Requested vote from peers 14c649e56e32428798b4895763179e23 (127.10.24.189:41319), 3609a8f04c864dbeb9e91fff62ecc4ca (127.10.24.188:37777)
I20250904 19:58:53.193552 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:53.193257 16557 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de9343d4ee8749d5b98effa595ef18e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca"
I20250904 19:58:53.194187 16557 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:53.193280 16492 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "de9343d4ee8749d5b98effa595ef18e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14c649e56e32428798b4895763179e23"
I20250904 19:58:53.195901 16492 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:53.196602 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:53.200034 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:53.200251 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:53.200515 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:53.200670 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:53.202534 16557 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de9343d4ee8749d5b98effa595ef18e3 in term 1.
I20250904 19:58:53.204200 16492 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate de9343d4ee8749d5b98effa595ef18e3 in term 1.
I20250904 19:58:53.205904 16403 leader_election.cc:304] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14c649e56e32428798b4895763179e23, de9343d4ee8749d5b98effa595ef18e3; no voters:
I20250904 19:58:53.208745 16613 raft_consensus.cc:2802] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:53.210091 16613 raft_consensus.cc:695] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [term 1 LEADER]: Becoming Leader. State: Replica: de9343d4ee8749d5b98effa595ef18e3, State: Running, Role: LEADER
I20250904 19:58:53.211171 16613 consensus_queue.cc:237] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } }
I20250904 19:58:53.215288 16719 sys_catalog.cc:455] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de9343d4ee8749d5b98effa595ef18e3. Latest consensus state: current_term: 1 leader_uuid: "de9343d4ee8749d5b98effa595ef18e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } } }
I20250904 19:58:53.216176 16719 sys_catalog.cc:458] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:53.218411 16727 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 19:58:53.226922 16727 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 19:58:53.239921 16557 raft_consensus.cc:1273] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [term 1 FOLLOWER]: Refusing update from remote peer de9343d4ee8749d5b98effa595ef18e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:53.240304 16492 raft_consensus.cc:1273] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [term 1 FOLLOWER]: Refusing update from remote peer de9343d4ee8749d5b98effa595ef18e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 19:58:53.241539 16719 consensus_queue.cc:1035] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:53.242242 16613 consensus_queue.cc:1035] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:53.252036 16587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader de9343d4ee8749d5b98effa595ef18e3. Latest consensus state: current_term: 1 leader_uuid: "de9343d4ee8749d5b98effa595ef18e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } } }
I20250904 19:58:53.252664 16587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:53.261592 16613 sys_catalog.cc:455] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "de9343d4ee8749d5b98effa595ef18e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } } }
I20250904 19:58:53.262432 16613 sys_catalog.cc:458] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:53.264777 16600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de9343d4ee8749d5b98effa595ef18e3. Latest consensus state: current_term: 1 leader_uuid: "de9343d4ee8749d5b98effa595ef18e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } } }
I20250904 19:58:53.265545 16600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:53.270713 16727 catalog_manager.cc:1349] Generated new cluster ID: 358caa29654e4b63b00181d648cf2e92
I20250904 19:58:53.271051 16727 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 19:58:53.273240 16719 sys_catalog.cc:455] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "de9343d4ee8749d5b98effa595ef18e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } } }
I20250904 19:58:53.273929 16719 sys_catalog.cc:458] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [sys.catalog]: This master's current role is: LEADER
I20250904 19:58:53.281702 16600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "de9343d4ee8749d5b98effa595ef18e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } } }
I20250904 19:58:53.282481 16600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:53.289908 16587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "de9343d4ee8749d5b98effa595ef18e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } } }
I20250904 19:58:53.291986 16587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:53.314445 16727 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 19:58:53.316462 16727 catalog_manager.cc:1506] Loading token signing keys...
I20250904 19:58:53.330365 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.130:41795
I20250904 19:58:53.330590 16785 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.130:41795 every 8 connection(s)
I20250904 19:58:53.357407 16727 catalog_manager.cc:5955] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3: Generated new TSK 0
I20250904 19:58:53.358348 16727 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 19:58:53.377609 16786 heartbeater.cc:344] Connected to a master server at 127.10.24.188:37777
I20250904 19:58:53.378124 16786 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:53.379146 16786 heartbeater.cc:507] Master 127.10.24.188:37777 requested a full tablet report, sending...
I20250904 19:58:53.381287 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:53.385154 16546 ts_manager.cc:194] Registered new tserver with Master: 1121c8a9c5f64923856f52dd7c39b015 (127.10.24.130:41795)
I20250904 19:58:53.390669 16417 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55778
I20250904 19:58:53.396369 16787 heartbeater.cc:344] Connected to a master server at 127.10.24.189:41319
I20250904 19:58:53.396853 16787 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:53.397876 16787 heartbeater.cc:507] Master 127.10.24.189:41319 requested a full tablet report, sending...
W20250904 19:58:53.400315 16795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:53.400928 16481 ts_manager.cc:194] Registered new tserver with Master: 1121c8a9c5f64923856f52dd7c39b015 (127.10.24.130:41795)
W20250904 19:58:53.402271 16794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:53.404254 16790 heartbeater.cc:344] Connected to a master server at 127.10.24.190:43389
I20250904 19:58:53.404692 16790 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:53.405628 16790 heartbeater.cc:507] Master 127.10.24.190:43389 requested a full tablet report, sending...
W20250904 19:58:53.405639 16797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:53.408231 16417 ts_manager.cc:194] Registered new tserver with Master: 1121c8a9c5f64923856f52dd7c39b015 (127.10.24.130:41795)
I20250904 19:58:53.408634 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:53.409749 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:53.410043 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:53.410264 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015933410239 us; error 0 us; skew 500 ppm
I20250904 19:58:53.410810 16698 heartbeater.cc:499] Master 127.10.24.190:43389 was elected leader, sending a full tablet report...
I20250904 19:58:53.411062 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:53.411795 16417 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55786
I20250904 19:58:53.416813 10338 webserver.cc:480] Webserver started at http://127.10.24.131:39635/ using document root <none> and password file <none>
I20250904 19:58:53.417407 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:53.417596 10338 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 19:58:53.417836 10338 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 19:58:53.419099 10338 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/ts-2-root/instance:
uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51"
format_stamp: "Formatted at 2025-09-04 19:58:53 on dist-test-slave-f3xj"
I20250904 19:58:53.423777 10338 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20250904 19:58:53.426515 16790 heartbeater.cc:499] Master 127.10.24.190:43389 was elected leader, sending a full tablet report...
I20250904 19:58:53.430655 16802 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:53.431594 10338 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20250904 19:58:53.431926 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/ts-2-root
uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51"
format_stamp: "Formatted at 2025-09-04 19:58:53 on dist-test-slave-f3xj"
I20250904 19:58:53.432211 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:53.450330 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:53.451714 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:53.453544 10338 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 19:58:53.457722 10338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 19:58:53.457957 10338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:53.458227 10338 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 19:58:53.458405 10338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:53.539814 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.131:41725
I20250904 19:58:53.539940 16864 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.131:41725 every 8 connection(s)
I20250904 19:58:53.591655 16865 heartbeater.cc:344] Connected to a master server at 127.10.24.188:37777
I20250904 19:58:53.592129 16865 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:53.593011 16865 heartbeater.cc:507] Master 127.10.24.188:37777 requested a full tablet report, sending...
I20250904 19:58:53.594024 16866 heartbeater.cc:344] Connected to a master server at 127.10.24.189:41319
I20250904 19:58:53.594470 16866 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:53.595386 16866 heartbeater.cc:507] Master 127.10.24.189:41319 requested a full tablet report, sending...
I20250904 19:58:53.596241 16546 ts_manager.cc:194] Registered new tserver with Master: f34422a0cbdc4389a55bd0e2b2cd3c51 (127.10.24.131:41725)
I20250904 19:58:53.599092 16482 ts_manager.cc:194] Registered new tserver with Master: f34422a0cbdc4389a55bd0e2b2cd3c51 (127.10.24.131:41725)
I20250904 19:58:53.602150 16868 heartbeater.cc:344] Connected to a master server at 127.10.24.190:43389
I20250904 19:58:53.602586 16868 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:53.603477 16868 heartbeater.cc:507] Master 127.10.24.190:43389 requested a full tablet report, sending...
I20250904 19:58:53.605288 16417 ts_manager.cc:194] Registered new tserver with Master: f34422a0cbdc4389a55bd0e2b2cd3c51 (127.10.24.131:41725)
I20250904 19:58:53.605525 10338 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.038781306s
I20250904 19:58:53.606652 16417 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55794
I20250904 19:58:53.624293 16868 heartbeater.cc:499] Master 127.10.24.190:43389 was elected leader, sending a full tablet report...
I20250904 19:58:53.673636 16414 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55796:
name: "ksck-test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\n\000\000\000\004\001\024\000\000\000""\004\001\n\000\000\000\004\001\024\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 19:58:53.676707 16414 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table ksck-test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 19:58:53.798254 16661 tablet_service.cc:1468] Processing CreateTablet for tablet 4cb1ebe48d6143cfa334fd0815ccf774 (DEFAULT_TABLE table=ksck-test-table [id=e257cb90ab9d4979a80fcf9678a05ea7]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:53.799706 16659 tablet_service.cc:1468] Processing CreateTablet for tablet a8831e0b2cef4873be73398ae7fef7e4 (DEFAULT_TABLE table=ksck-test-table [id=e257cb90ab9d4979a80fcf9678a05ea7]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:53.799731 16661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cb1ebe48d6143cfa334fd0815ccf774. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:53.801427 16660 tablet_service.cc:1468] Processing CreateTablet for tablet bc9d9ca580564ff4b7dc7921d0b16516 (DEFAULT_TABLE table=ksck-test-table [id=e257cb90ab9d4979a80fcf9678a05ea7]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:53.802853 16660 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc9d9ca580564ff4b7dc7921d0b16516. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:53.804760 16830 tablet_service.cc:1468] Processing CreateTablet for tablet 4cb1ebe48d6143cfa334fd0815ccf774 (DEFAULT_TABLE table=ksck-test-table [id=e257cb90ab9d4979a80fcf9678a05ea7]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:53.805276 16828 tablet_service.cc:1468] Processing CreateTablet for tablet a8831e0b2cef4873be73398ae7fef7e4 (DEFAULT_TABLE table=ksck-test-table [id=e257cb90ab9d4979a80fcf9678a05ea7]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:53.806603 16828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8831e0b2cef4873be73398ae7fef7e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:53.806926 16829 tablet_service.cc:1468] Processing CreateTablet for tablet bc9d9ca580564ff4b7dc7921d0b16516 (DEFAULT_TABLE table=ksck-test-table [id=e257cb90ab9d4979a80fcf9678a05ea7]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:53.813678 16829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc9d9ca580564ff4b7dc7921d0b16516. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:53.816514 16659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8831e0b2cef4873be73398ae7fef7e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:53.819983 16830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cb1ebe48d6143cfa334fd0815ccf774. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:53.835067 16743 tablet_service.cc:1468] Processing CreateTablet for tablet 4cb1ebe48d6143cfa334fd0815ccf774 (DEFAULT_TABLE table=ksck-test-table [id=e257cb90ab9d4979a80fcf9678a05ea7]), partition=RANGE (key) PARTITION VALUES < 10
I20250904 19:58:53.835399 16742 tablet_service.cc:1468] Processing CreateTablet for tablet bc9d9ca580564ff4b7dc7921d0b16516 (DEFAULT_TABLE table=ksck-test-table [id=e257cb90ab9d4979a80fcf9678a05ea7]), partition=RANGE (key) PARTITION 10 <= VALUES < 20
I20250904 19:58:53.836489 16743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cb1ebe48d6143cfa334fd0815ccf774. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:53.837299 16742 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc9d9ca580564ff4b7dc7921d0b16516. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:53.844669 16741 tablet_service.cc:1468] Processing CreateTablet for tablet a8831e0b2cef4873be73398ae7fef7e4 (DEFAULT_TABLE table=ksck-test-table [id=e257cb90ab9d4979a80fcf9678a05ea7]), partition=RANGE (key) PARTITION 20 <= VALUES
I20250904 19:58:53.846136 16741 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8831e0b2cef4873be73398ae7fef7e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 19:58:53.900632 16886 tablet_bootstrap.cc:492] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51: Bootstrap starting.
I20250904 19:58:53.914703 16886 tablet_bootstrap.cc:654] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:53.919705 16887 tablet_bootstrap.cc:492] T a8831e0b2cef4873be73398ae7fef7e4 P 34fcdc21eb7e477ebab2618592e9c276: Bootstrap starting.
I20250904 19:58:53.925436 16888 tablet_bootstrap.cc:492] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015: Bootstrap starting.
I20250904 19:58:53.938660 16886 tablet_bootstrap.cc:492] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51: No bootstrap required, opened a new log
I20250904 19:58:53.939342 16886 ts_tablet_manager.cc:1397] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51: Time spent bootstrapping tablet: real 0.039s user 0.010s sys 0.016s
I20250904 19:58:53.940773 16888 tablet_bootstrap.cc:654] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:53.944753 16887 tablet_bootstrap.cc:654] T a8831e0b2cef4873be73398ae7fef7e4 P 34fcdc21eb7e477ebab2618592e9c276: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:53.949999 16886 raft_consensus.cc:357] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:53.951026 16886 raft_consensus.cc:383] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:53.951404 16886 raft_consensus.cc:738] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f34422a0cbdc4389a55bd0e2b2cd3c51, State: Initialized, Role: FOLLOWER
I20250904 19:58:53.953321 16886 consensus_queue.cc:260] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:53.958531 16887 tablet_bootstrap.cc:492] T a8831e0b2cef4873be73398ae7fef7e4 P 34fcdc21eb7e477ebab2618592e9c276: No bootstrap required, opened a new log
I20250904 19:58:53.958729 16888 tablet_bootstrap.cc:492] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015: No bootstrap required, opened a new log
I20250904 19:58:53.959252 16887 ts_tablet_manager.cc:1397] T a8831e0b2cef4873be73398ae7fef7e4 P 34fcdc21eb7e477ebab2618592e9c276: Time spent bootstrapping tablet: real 0.040s user 0.015s sys 0.000s
I20250904 19:58:53.959441 16888 ts_tablet_manager.cc:1397] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015: Time spent bootstrapping tablet: real 0.034s user 0.018s sys 0.005s
I20250904 19:58:53.962884 16888 raft_consensus.cc:357] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:53.963750 16888 raft_consensus.cc:383] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:53.964131 16888 raft_consensus.cc:738] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1121c8a9c5f64923856f52dd7c39b015, State: Initialized, Role: FOLLOWER
I20250904 19:58:53.964869 16888 consensus_queue.cc:260] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:53.968560 16886 ts_tablet_manager.cc:1428] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51: Time spent starting tablet: real 0.029s user 0.009s sys 0.004s
I20250904 19:58:53.972947 16886 tablet_bootstrap.cc:492] T bc9d9ca580564ff4b7dc7921d0b16516 P f34422a0cbdc4389a55bd0e2b2cd3c51: Bootstrap starting.
I20250904 19:58:53.975931 16887 raft_consensus.cc:357] T a8831e0b2cef4873be73398ae7fef7e4 P 34fcdc21eb7e477ebab2618592e9c276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:53.976840 16887 raft_consensus.cc:383] T a8831e0b2cef4873be73398ae7fef7e4 P 34fcdc21eb7e477ebab2618592e9c276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:53.977231 16887 raft_consensus.cc:738] T a8831e0b2cef4873be73398ae7fef7e4 P 34fcdc21eb7e477ebab2618592e9c276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fcdc21eb7e477ebab2618592e9c276, State: Initialized, Role: FOLLOWER
I20250904 19:58:53.974272 16888 ts_tablet_manager.cc:1428] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015: Time spent starting tablet: real 0.014s user 0.002s sys 0.005s
I20250904 19:58:53.978813 16888 tablet_bootstrap.cc:492] T 4cb1ebe48d6143cfa334fd0815ccf774 P 1121c8a9c5f64923856f52dd7c39b015: Bootstrap starting.
I20250904 19:58:53.980579 16887 consensus_queue.cc:260] T a8831e0b2cef4873be73398ae7fef7e4 P 34fcdc21eb7e477ebab2618592e9c276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:53.986159 16886 tablet_bootstrap.cc:654] T bc9d9ca580564ff4b7dc7921d0b16516 P f34422a0cbdc4389a55bd0e2b2cd3c51: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:53.991758 16887 ts_tablet_manager.cc:1428] T a8831e0b2cef4873be73398ae7fef7e4 P 34fcdc21eb7e477ebab2618592e9c276: Time spent starting tablet: real 0.032s user 0.001s sys 0.012s
I20250904 19:58:53.993098 16888 tablet_bootstrap.cc:654] T 4cb1ebe48d6143cfa334fd0815ccf774 P 1121c8a9c5f64923856f52dd7c39b015: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:53.994871 16887 tablet_bootstrap.cc:492] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276: Bootstrap starting.
I20250904 19:58:54.006656 16599 catalog_manager.cc:1261] Loaded cluster ID: 358caa29654e4b63b00181d648cf2e92
I20250904 19:58:54.007081 16599 catalog_manager.cc:1554] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca: loading cluster ID for follower catalog manager: success
I20250904 19:58:54.021389 16599 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca: acquiring CA information for follower catalog manager: success
I20250904 19:58:54.024958 16887 tablet_bootstrap.cc:654] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:54.025437 16612 catalog_manager.cc:1261] Loaded cluster ID: 358caa29654e4b63b00181d648cf2e92
I20250904 19:58:54.028461 16612 catalog_manager.cc:1554] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23: loading cluster ID for follower catalog manager: success
I20250904 19:58:54.027365 16893 raft_consensus.cc:491] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:54.035382 16599 catalog_manager.cc:1604] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:58:54.035892 16893 raft_consensus.cc:513] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.038568 16888 tablet_bootstrap.cc:492] T 4cb1ebe48d6143cfa334fd0815ccf774 P 1121c8a9c5f64923856f52dd7c39b015: No bootstrap required, opened a new log
I20250904 19:58:54.038583 16886 tablet_bootstrap.cc:492] T bc9d9ca580564ff4b7dc7921d0b16516 P f34422a0cbdc4389a55bd0e2b2cd3c51: No bootstrap required, opened a new log
I20250904 19:58:54.041499 16886 ts_tablet_manager.cc:1397] T bc9d9ca580564ff4b7dc7921d0b16516 P f34422a0cbdc4389a55bd0e2b2cd3c51: Time spent bootstrapping tablet: real 0.069s user 0.014s sys 0.012s
I20250904 19:58:54.035367 16612 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23: acquiring CA information for follower catalog manager: success
I20250904 19:58:54.039181 16888 ts_tablet_manager.cc:1397] T 4cb1ebe48d6143cfa334fd0815ccf774 P 1121c8a9c5f64923856f52dd7c39b015: Time spent bootstrapping tablet: real 0.061s user 0.011s sys 0.013s
I20250904 19:58:54.049260 16888 raft_consensus.cc:357] T 4cb1ebe48d6143cfa334fd0815ccf774 P 1121c8a9c5f64923856f52dd7c39b015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.049993 16888 raft_consensus.cc:383] T 4cb1ebe48d6143cfa334fd0815ccf774 P 1121c8a9c5f64923856f52dd7c39b015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:54.050310 16888 raft_consensus.cc:738] T 4cb1ebe48d6143cfa334fd0815ccf774 P 1121c8a9c5f64923856f52dd7c39b015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1121c8a9c5f64923856f52dd7c39b015, State: Initialized, Role: FOLLOWER
I20250904 19:58:54.051043 16888 consensus_queue.cc:260] T 4cb1ebe48d6143cfa334fd0815ccf774 P 1121c8a9c5f64923856f52dd7c39b015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.058719 16886 raft_consensus.cc:357] T bc9d9ca580564ff4b7dc7921d0b16516 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.059551 16886 raft_consensus.cc:383] T bc9d9ca580564ff4b7dc7921d0b16516 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:54.059866 16886 raft_consensus.cc:738] T bc9d9ca580564ff4b7dc7921d0b16516 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f34422a0cbdc4389a55bd0e2b2cd3c51, State: Initialized, Role: FOLLOWER
I20250904 19:58:54.061522 16887 tablet_bootstrap.cc:492] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276: No bootstrap required, opened a new log
I20250904 19:58:54.062090 16887 ts_tablet_manager.cc:1397] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276: Time spent bootstrapping tablet: real 0.067s user 0.012s sys 0.001s
I20250904 19:58:54.063992 16886 consensus_queue.cc:260] T bc9d9ca580564ff4b7dc7921d0b16516 P f34422a0cbdc4389a55bd0e2b2cd3c51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
W20250904 19:58:54.067281 16871 tablet.cc:2378] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250904 19:58:54.068694 16886 ts_tablet_manager.cc:1428] T bc9d9ca580564ff4b7dc7921d0b16516 P f34422a0cbdc4389a55bd0e2b2cd3c51: Time spent starting tablet: real 0.027s user 0.006s sys 0.000s
I20250904 19:58:54.069821 16886 tablet_bootstrap.cc:492] T a8831e0b2cef4873be73398ae7fef7e4 P f34422a0cbdc4389a55bd0e2b2cd3c51: Bootstrap starting.
I20250904 19:58:54.066229 16887 raft_consensus.cc:357] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.072975 16887 raft_consensus.cc:383] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:54.073386 16887 raft_consensus.cc:738] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fcdc21eb7e477ebab2618592e9c276, State: Initialized, Role: FOLLOWER
I20250904 19:58:54.074193 16887 consensus_queue.cc:260] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.056124 16893 leader_election.cc:290] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fcdc21eb7e477ebab2618592e9c276 (127.10.24.129:46843), f34422a0cbdc4389a55bd0e2b2cd3c51 (127.10.24.131:41725)
I20250904 19:58:54.076970 16887 ts_tablet_manager.cc:1428] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276: Time spent starting tablet: real 0.015s user 0.003s sys 0.003s
I20250904 19:58:54.082162 16887 tablet_bootstrap.cc:492] T 4cb1ebe48d6143cfa334fd0815ccf774 P 34fcdc21eb7e477ebab2618592e9c276: Bootstrap starting.
I20250904 19:58:54.086897 16886 tablet_bootstrap.cc:654] T a8831e0b2cef4873be73398ae7fef7e4 P f34422a0cbdc4389a55bd0e2b2cd3c51: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:54.107139 16888 ts_tablet_manager.cc:1428] T 4cb1ebe48d6143cfa334fd0815ccf774 P 1121c8a9c5f64923856f52dd7c39b015: Time spent starting tablet: real 0.061s user 0.015s sys 0.005s
I20250904 19:58:54.109664 16892 raft_consensus.cc:491] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:54.110558 16892 raft_consensus.cc:513] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.114604 16892 leader_election.cc:290] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34fcdc21eb7e477ebab2618592e9c276 (127.10.24.129:46843), 1121c8a9c5f64923856f52dd7c39b015 (127.10.24.130:41795)
I20250904 19:58:54.158990 16612 catalog_manager.cc:1604] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:58:54.158764 16888 tablet_bootstrap.cc:492] T bc9d9ca580564ff4b7dc7921d0b16516 P 1121c8a9c5f64923856f52dd7c39b015: Bootstrap starting.
I20250904 19:58:54.171141 16671 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a8831e0b2cef4873be73398ae7fef7e4" candidate_uuid: "1121c8a9c5f64923856f52dd7c39b015" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fcdc21eb7e477ebab2618592e9c276" is_pre_election: true
I20250904 19:58:54.171900 16671 raft_consensus.cc:2466] T a8831e0b2cef4873be73398ae7fef7e4 P 34fcdc21eb7e477ebab2618592e9c276 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1121c8a9c5f64923856f52dd7c39b015 in term 0.
I20250904 19:58:54.178905 16713 leader_election.cc:304] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1121c8a9c5f64923856f52dd7c39b015, 34fcdc21eb7e477ebab2618592e9c276; no voters:
I20250904 19:58:54.179701 16886 tablet_bootstrap.cc:492] T a8831e0b2cef4873be73398ae7fef7e4 P f34422a0cbdc4389a55bd0e2b2cd3c51: No bootstrap required, opened a new log
I20250904 19:58:54.180258 16900 raft_consensus.cc:2802] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:54.182683 16900 raft_consensus.cc:491] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:54.186450 16900 raft_consensus.cc:3058] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:54.180251 16886 ts_tablet_manager.cc:1397] T a8831e0b2cef4873be73398ae7fef7e4 P f34422a0cbdc4389a55bd0e2b2cd3c51: Time spent bootstrapping tablet: real 0.111s user 0.014s sys 0.016s
I20250904 19:58:54.191418 16886 raft_consensus.cc:357] T a8831e0b2cef4873be73398ae7fef7e4 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.195219 16888 tablet_bootstrap.cc:654] T bc9d9ca580564ff4b7dc7921d0b16516 P 1121c8a9c5f64923856f52dd7c39b015: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:54.195418 16887 tablet_bootstrap.cc:654] T 4cb1ebe48d6143cfa334fd0815ccf774 P 34fcdc21eb7e477ebab2618592e9c276: Neither blocks nor log segments found. Creating new log.
I20250904 19:58:54.197670 16886 raft_consensus.cc:383] T a8831e0b2cef4873be73398ae7fef7e4 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:54.199393 16886 raft_consensus.cc:738] T a8831e0b2cef4873be73398ae7fef7e4 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f34422a0cbdc4389a55bd0e2b2cd3c51, State: Initialized, Role: FOLLOWER
I20250904 19:58:54.203234 16840 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a8831e0b2cef4873be73398ae7fef7e4" candidate_uuid: "1121c8a9c5f64923856f52dd7c39b015" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" is_pre_election: true
I20250904 19:58:54.209883 16671 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4cb1ebe48d6143cfa334fd0815ccf774" candidate_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fcdc21eb7e477ebab2618592e9c276" is_pre_election: true
W20250904 19:58:54.212177 16804 leader_election.cc:343] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34fcdc21eb7e477ebab2618592e9c276 (127.10.24.129:46843): Illegal state: must be running to vote when last-logged opid is not known
I20250904 19:58:54.200415 16886 consensus_queue.cc:260] T a8831e0b2cef4873be73398ae7fef7e4 P f34422a0cbdc4389a55bd0e2b2cd3c51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.213196 16840 raft_consensus.cc:2466] T a8831e0b2cef4873be73398ae7fef7e4 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1121c8a9c5f64923856f52dd7c39b015 in term 0.
I20250904 19:58:54.240447 16756 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4cb1ebe48d6143cfa334fd0815ccf774" candidate_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1121c8a9c5f64923856f52dd7c39b015" is_pre_election: true
I20250904 19:58:54.241192 16756 raft_consensus.cc:2466] T 4cb1ebe48d6143cfa334fd0815ccf774 P 1121c8a9c5f64923856f52dd7c39b015 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f34422a0cbdc4389a55bd0e2b2cd3c51 in term 0.
I20250904 19:58:54.245661 16803 leader_election.cc:304] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1121c8a9c5f64923856f52dd7c39b015, f34422a0cbdc4389a55bd0e2b2cd3c51; no voters: 34fcdc21eb7e477ebab2618592e9c276
I20250904 19:58:54.246914 16892 raft_consensus.cc:2802] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:54.247295 16892 raft_consensus.cc:491] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:54.247639 16892 raft_consensus.cc:3058] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:54.260286 16892 raft_consensus.cc:513] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.271695 16886 ts_tablet_manager.cc:1428] T a8831e0b2cef4873be73398ae7fef7e4 P f34422a0cbdc4389a55bd0e2b2cd3c51: Time spent starting tablet: real 0.083s user 0.005s sys 0.003s
I20250904 19:58:54.272019 16900 raft_consensus.cc:513] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.286777 16671 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4cb1ebe48d6143cfa334fd0815ccf774" candidate_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fcdc21eb7e477ebab2618592e9c276"
W20250904 19:58:54.288571 16804 leader_election.cc:343] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 34fcdc21eb7e477ebab2618592e9c276 (127.10.24.129:46843): Illegal state: must be running to vote when last-logged opid is not known
I20250904 19:58:54.286808 16756 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4cb1ebe48d6143cfa334fd0815ccf774" candidate_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1121c8a9c5f64923856f52dd7c39b015"
I20250904 19:58:54.289714 16671 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a8831e0b2cef4873be73398ae7fef7e4" candidate_uuid: "1121c8a9c5f64923856f52dd7c39b015" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34fcdc21eb7e477ebab2618592e9c276"
I20250904 19:58:54.289990 16756 raft_consensus.cc:3058] T 4cb1ebe48d6143cfa334fd0815ccf774 P 1121c8a9c5f64923856f52dd7c39b015 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:54.290352 16671 raft_consensus.cc:3058] T a8831e0b2cef4873be73398ae7fef7e4 P 34fcdc21eb7e477ebab2618592e9c276 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:54.301177 16756 raft_consensus.cc:2466] T 4cb1ebe48d6143cfa334fd0815ccf774 P 1121c8a9c5f64923856f52dd7c39b015 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f34422a0cbdc4389a55bd0e2b2cd3c51 in term 1.
I20250904 19:58:54.302675 16803 leader_election.cc:304] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1121c8a9c5f64923856f52dd7c39b015, f34422a0cbdc4389a55bd0e2b2cd3c51; no voters: 34fcdc21eb7e477ebab2618592e9c276
I20250904 19:58:54.310362 16892 leader_election.cc:290] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [CANDIDATE]: Term 1 election: Requested vote from peers 34fcdc21eb7e477ebab2618592e9c276 (127.10.24.129:46843), 1121c8a9c5f64923856f52dd7c39b015 (127.10.24.130:41795)
I20250904 19:58:54.312762 16887 tablet_bootstrap.cc:492] T 4cb1ebe48d6143cfa334fd0815ccf774 P 34fcdc21eb7e477ebab2618592e9c276: No bootstrap required, opened a new log
I20250904 19:58:54.312542 16840 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a8831e0b2cef4873be73398ae7fef7e4" candidate_uuid: "1121c8a9c5f64923856f52dd7c39b015" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51"
I20250904 19:58:54.313011 16892 raft_consensus.cc:2802] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:54.313503 16900 leader_election.cc:290] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015 [CANDIDATE]: Term 1 election: Requested vote from peers 34fcdc21eb7e477ebab2618592e9c276 (127.10.24.129:46843), f34422a0cbdc4389a55bd0e2b2cd3c51 (127.10.24.131:41725)
I20250904 19:58:54.313707 16894 raft_consensus.cc:491] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:54.314894 16892 raft_consensus.cc:695] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 1 LEADER]: Becoming Leader. State: Replica: f34422a0cbdc4389a55bd0e2b2cd3c51, State: Running, Role: LEADER
I20250904 19:58:54.315300 16894 raft_consensus.cc:513] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.316140 16892 consensus_queue.cc:237] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.317850 16894 leader_election.cc:290] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1121c8a9c5f64923856f52dd7c39b015 (127.10.24.130:41795), f34422a0cbdc4389a55bd0e2b2cd3c51 (127.10.24.131:41725)
I20250904 19:58:54.314208 16840 raft_consensus.cc:3058] T a8831e0b2cef4873be73398ae7fef7e4 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:54.314874 16887 ts_tablet_manager.cc:1397] T 4cb1ebe48d6143cfa334fd0815ccf774 P 34fcdc21eb7e477ebab2618592e9c276: Time spent bootstrapping tablet: real 0.233s user 0.049s sys 0.046s
I20250904 19:58:54.322922 16887 raft_consensus.cc:357] T 4cb1ebe48d6143cfa334fd0815ccf774 P 34fcdc21eb7e477ebab2618592e9c276 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.323809 16887 raft_consensus.cc:383] T 4cb1ebe48d6143cfa334fd0815ccf774 P 34fcdc21eb7e477ebab2618592e9c276 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:54.324199 16887 raft_consensus.cc:738] T 4cb1ebe48d6143cfa334fd0815ccf774 P 34fcdc21eb7e477ebab2618592e9c276 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34fcdc21eb7e477ebab2618592e9c276, State: Initialized, Role: FOLLOWER
I20250904 19:58:54.324975 16887 consensus_queue.cc:260] T 4cb1ebe48d6143cfa334fd0815ccf774 P 34fcdc21eb7e477ebab2618592e9c276 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.327620 16887 ts_tablet_manager.cc:1428] T 4cb1ebe48d6143cfa334fd0815ccf774 P 34fcdc21eb7e477ebab2618592e9c276: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20250904 19:58:54.338912 16840 raft_consensus.cc:2466] T a8831e0b2cef4873be73398ae7fef7e4 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1121c8a9c5f64923856f52dd7c39b015 in term 1.
I20250904 19:58:54.314285 16671 raft_consensus.cc:2466] T a8831e0b2cef4873be73398ae7fef7e4 P 34fcdc21eb7e477ebab2618592e9c276 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1121c8a9c5f64923856f52dd7c39b015 in term 1.
I20250904 19:58:54.343886 16713 leader_election.cc:304] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1121c8a9c5f64923856f52dd7c39b015, 34fcdc21eb7e477ebab2618592e9c276; no voters:
W20250904 19:58:54.347970 16803 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.10.24.130:41795, user_credentials={real_user=slave}} blocked reactor thread for 45392.9us
I20250904 19:58:54.361562 16900 raft_consensus.cc:2802] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:54.363540 16900 raft_consensus.cc:695] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015 [term 1 LEADER]: Becoming Leader. State: Replica: 1121c8a9c5f64923856f52dd7c39b015, State: Running, Role: LEADER
I20250904 19:58:54.365446 16900 consensus_queue.cc:237] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.369640 16417 catalog_manager.cc:5582] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 reported cstate change: term changed from 0 to 1, leader changed from <none> to f34422a0cbdc4389a55bd0e2b2cd3c51 (127.10.24.131). New cstate: current_term: 1 leader_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } health_report { overall_health: HEALTHY } } }
I20250904 19:58:54.378438 16888 tablet_bootstrap.cc:492] T bc9d9ca580564ff4b7dc7921d0b16516 P 1121c8a9c5f64923856f52dd7c39b015: No bootstrap required, opened a new log
I20250904 19:58:54.378988 16888 ts_tablet_manager.cc:1397] T bc9d9ca580564ff4b7dc7921d0b16516 P 1121c8a9c5f64923856f52dd7c39b015: Time spent bootstrapping tablet: real 0.228s user 0.049s sys 0.095s
I20250904 19:58:54.383081 16888 raft_consensus.cc:357] T bc9d9ca580564ff4b7dc7921d0b16516 P 1121c8a9c5f64923856f52dd7c39b015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.383808 16888 raft_consensus.cc:383] T bc9d9ca580564ff4b7dc7921d0b16516 P 1121c8a9c5f64923856f52dd7c39b015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 19:58:54.384099 16888 raft_consensus.cc:738] T bc9d9ca580564ff4b7dc7921d0b16516 P 1121c8a9c5f64923856f52dd7c39b015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1121c8a9c5f64923856f52dd7c39b015, State: Initialized, Role: FOLLOWER
I20250904 19:58:54.384873 16888 consensus_queue.cc:260] T bc9d9ca580564ff4b7dc7921d0b16516 P 1121c8a9c5f64923856f52dd7c39b015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.391991 16888 ts_tablet_manager.cc:1428] T bc9d9ca580564ff4b7dc7921d0b16516 P 1121c8a9c5f64923856f52dd7c39b015: Time spent starting tablet: real 0.013s user 0.009s sys 0.000s
I20250904 19:58:54.398401 16756 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bc9d9ca580564ff4b7dc7921d0b16516" candidate_uuid: "34fcdc21eb7e477ebab2618592e9c276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1121c8a9c5f64923856f52dd7c39b015" is_pre_election: true
I20250904 19:58:54.399087 16756 raft_consensus.cc:2466] T bc9d9ca580564ff4b7dc7921d0b16516 P 1121c8a9c5f64923856f52dd7c39b015 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fcdc21eb7e477ebab2618592e9c276 in term 0.
I20250904 19:58:54.400400 16840 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bc9d9ca580564ff4b7dc7921d0b16516" candidate_uuid: "34fcdc21eb7e477ebab2618592e9c276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" is_pre_election: true
I20250904 19:58:54.400926 16840 raft_consensus.cc:2466] T bc9d9ca580564ff4b7dc7921d0b16516 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34fcdc21eb7e477ebab2618592e9c276 in term 0.
I20250904 19:58:54.401892 16634 leader_election.cc:304] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1121c8a9c5f64923856f52dd7c39b015, 34fcdc21eb7e477ebab2618592e9c276; no voters:
I20250904 19:58:54.402709 16894 raft_consensus.cc:2802] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 19:58:54.403050 16894 raft_consensus.cc:491] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 19:58:54.403363 16894 raft_consensus.cc:3058] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:54.412372 16894 raft_consensus.cc:513] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.414538 16894 leader_election.cc:290] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276 [CANDIDATE]: Term 1 election: Requested vote from peers 1121c8a9c5f64923856f52dd7c39b015 (127.10.24.130:41795), f34422a0cbdc4389a55bd0e2b2cd3c51 (127.10.24.131:41725)
I20250904 19:58:54.423102 16756 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bc9d9ca580564ff4b7dc7921d0b16516" candidate_uuid: "34fcdc21eb7e477ebab2618592e9c276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1121c8a9c5f64923856f52dd7c39b015"
I20250904 19:58:54.423353 16840 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bc9d9ca580564ff4b7dc7921d0b16516" candidate_uuid: "34fcdc21eb7e477ebab2618592e9c276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51"
I20250904 19:58:54.423758 16756 raft_consensus.cc:3058] T bc9d9ca580564ff4b7dc7921d0b16516 P 1121c8a9c5f64923856f52dd7c39b015 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:54.425879 16840 raft_consensus.cc:3058] T bc9d9ca580564ff4b7dc7921d0b16516 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:54.436771 16756 raft_consensus.cc:2466] T bc9d9ca580564ff4b7dc7921d0b16516 P 1121c8a9c5f64923856f52dd7c39b015 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fcdc21eb7e477ebab2618592e9c276 in term 1.
I20250904 19:58:54.444180 16634 leader_election.cc:304] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1121c8a9c5f64923856f52dd7c39b015, 34fcdc21eb7e477ebab2618592e9c276; no voters:
I20250904 19:58:54.448056 16840 raft_consensus.cc:2466] T bc9d9ca580564ff4b7dc7921d0b16516 P f34422a0cbdc4389a55bd0e2b2cd3c51 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34fcdc21eb7e477ebab2618592e9c276 in term 1.
I20250904 19:58:54.444867 16894 raft_consensus.cc:2802] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 19:58:54.457896 16894 raft_consensus.cc:695] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276 [term 1 LEADER]: Becoming Leader. State: Replica: 34fcdc21eb7e477ebab2618592e9c276, State: Running, Role: LEADER
I20250904 19:58:54.459427 16894 consensus_queue.cc:237] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.465225 16416 catalog_manager.cc:5582] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1121c8a9c5f64923856f52dd7c39b015 (127.10.24.130). New cstate: current_term: 1 leader_uuid: "1121c8a9c5f64923856f52dd7c39b015" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } health_report { overall_health: UNKNOWN } } }
I20250904 19:58:54.522246 16414 catalog_manager.cc:5582] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34fcdc21eb7e477ebab2618592e9c276 (127.10.24.129). New cstate: current_term: 1 leader_uuid: "34fcdc21eb7e477ebab2618592e9c276" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } health_report { overall_health: UNKNOWN } } }
I20250904 19:58:54.568332 16894 raft_consensus.cc:491] T 4cb1ebe48d6143cfa334fd0815ccf774 P 34fcdc21eb7e477ebab2618592e9c276 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 19:58:54.568871 16894 raft_consensus.cc:513] T 4cb1ebe48d6143cfa334fd0815ccf774 P 34fcdc21eb7e477ebab2618592e9c276 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 } } peers { permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 } } peers { permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 } }
I20250904 19:58:54.570787 16894 leader_election.cc:290] T 4cb1ebe48d6143cfa334fd0815ccf774 P 34fcdc21eb7e477ebab2618592e9c276 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1121c8a9c5f64923856f52dd7c39b015 (127.10.24.130:41795), f34422a0cbdc4389a55bd0e2b2cd3c51 (127.10.24.131:41725)
I20250904 19:58:54.574714 16840 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4cb1ebe48d6143cfa334fd0815ccf774" candidate_uuid: "34fcdc21eb7e477ebab2618592e9c276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" is_pre_election: true
I20250904 19:58:54.575601 16756 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4cb1ebe48d6143cfa334fd0815ccf774" candidate_uuid: "34fcdc21eb7e477ebab2618592e9c276" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1121c8a9c5f64923856f52dd7c39b015" is_pre_election: true
I20250904 19:58:54.576390 16756 raft_consensus.cc:2391] T 4cb1ebe48d6143cfa334fd0815ccf774 P 1121c8a9c5f64923856f52dd7c39b015 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 34fcdc21eb7e477ebab2618592e9c276 in current term 1: Already voted for candidate f34422a0cbdc4389a55bd0e2b2cd3c51 in this term.
I20250904 19:58:54.578508 16634 leader_election.cc:304] T 4cb1ebe48d6143cfa334fd0815ccf774 P 34fcdc21eb7e477ebab2618592e9c276 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 34fcdc21eb7e477ebab2618592e9c276; no voters: 1121c8a9c5f64923856f52dd7c39b015, f34422a0cbdc4389a55bd0e2b2cd3c51
I20250904 19:58:54.580504 16894 raft_consensus.cc:3058] T 4cb1ebe48d6143cfa334fd0815ccf774 P 34fcdc21eb7e477ebab2618592e9c276 [term 0 FOLLOWER]: Advancing to term 1
I20250904 19:58:54.588598 16894 raft_consensus.cc:2747] T 4cb1ebe48d6143cfa334fd0815ccf774 P 34fcdc21eb7e477ebab2618592e9c276 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250904 19:58:54.604983 10338 master.cc:561] Master@127.10.24.190:43389 shutting down...
I20250904 19:58:54.629022 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [term 1 LEADER]: Raft consensus shutting down.
I20250904 19:58:54.630237 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:54.634776 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3: stopping tablet replica
I20250904 19:58:54.653961 10338 master.cc:583] Master@127.10.24.190:43389 shutdown complete.
I20250904 19:58:54.670895 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 19:58:54.686468 16928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:54.688866 16929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:54.689832 10338 server_base.cc:1047] running on GCE node
W20250904 19:58:54.691241 16932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:54.692121 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:54.692350 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:54.692528 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015934692514 us; error 0 us; skew 500 ppm
I20250904 19:58:54.693064 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:54.699903 10338 webserver.cc:480] Webserver started at http://127.10.24.190:37987/ using document root <none> and password file <none>
I20250904 19:58:54.700474 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:54.700711 10338 fs_manager.cc:365] Using existing metadata directory in first data directory
I20250904 19:58:54.706005 10338 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20250904 19:58:54.709755 16936 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:54.713546 10338 fs_manager.cc:730] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20250904 19:58:54.713948 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-0-root
uuid: "de9343d4ee8749d5b98effa595ef18e3"
format_stamp: "Formatted at 2025-09-04 19:58:52 on dist-test-slave-f3xj"
I20250904 19:58:54.714321 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:54.740594 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:54.741870 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:54.744062 16907 consensus_queue.cc:1035] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:54.758388 16900 consensus_queue.cc:1035] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:54.798161 16907 consensus_queue.cc:1035] T 4cb1ebe48d6143cfa334fd0815ccf774 P f34422a0cbdc4389a55bd0e2b2cd3c51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:54.833194 16900 consensus_queue.cc:1035] T a8831e0b2cef4873be73398ae7fef7e4 P 1121c8a9c5f64923856f52dd7c39b015 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" member_type: VOTER last_known_addr { host: "127.10.24.129" port: 46843 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:54.913952 16894 consensus_queue.cc:1035] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" member_type: VOTER last_known_addr { host: "127.10.24.130" port: 41795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:54.983333 16894 consensus_queue.cc:1035] T bc9d9ca580564ff4b7dc7921d0b16516 P 34fcdc21eb7e477ebab2618592e9c276 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" member_type: VOTER last_known_addr { host: "127.10.24.131" port: 41725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 19:58:55.052138 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.190:43389
I20250904 19:58:55.052225 17001 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.190:43389 every 8 connection(s)
I20250904 19:58:55.055546 10338 master.cc:561] Master@127.10.24.189:41319 shutting down...
I20250904 19:58:55.071321 17002 sys_catalog.cc:263] Verifying existing consensus state
I20250904 19:58:55.085022 17002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3: Bootstrap starting.
I20250904 19:58:55.093858 16790 heartbeater.cc:344] Connected to a master server at 127.10.24.190:43389
I20250904 19:58:55.107908 16698 heartbeater.cc:344] Connected to a master server at 127.10.24.190:43389
I20250904 19:58:55.109848 16868 heartbeater.cc:344] Connected to a master server at 127.10.24.190:43389
I20250904 19:58:55.128892 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:55.129565 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:55.129953 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23: stopping tablet replica
I20250904 19:58:55.151584 17002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=7 ignored=0} mutations{seen=6 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250904 19:58:55.152649 17002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3: Bootstrap complete.
I20250904 19:58:55.155279 17002 raft_consensus.cc:357] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } }
I20250904 19:58:55.155998 17002 raft_consensus.cc:738] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: de9343d4ee8749d5b98effa595ef18e3, State: Initialized, Role: FOLLOWER
I20250904 19:58:55.156270 10338 master.cc:583] Master@127.10.24.189:41319 shutdown complete.
I20250904 19:58:55.157174 17002 consensus_queue.cc:260] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } }
I20250904 19:58:55.172847 16698 heartbeater.cc:344] Connected to a master server at 127.10.24.190:43389
I20250904 19:58:55.178623 10338 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20250904 19:58:55.178790 17008 sys_catalog.cc:455] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } } }
I20250904 19:58:55.179546 17008 sys_catalog.cc:458] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:55.182176 17002 sys_catalog.cc:564] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [sys.catalog]: configured and running, proceeding with master startup.
W20250904 19:58:55.184818 17010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:55.193264 17009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 19:58:55.197682 17014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 19:58:55.197933 10338 server_base.cc:1047] running on GCE node
I20250904 19:58:55.199020 10338 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 19:58:55.199259 10338 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 19:58:55.199491 10338 hybrid_clock.cc:648] HybridClock initialized: now 1757015935199470 us; error 0 us; skew 500 ppm
I20250904 19:58:55.200047 10338 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 19:58:55.203184 10338 webserver.cc:480] Webserver started at http://127.10.24.189:45973/ using document root <none> and password file <none>
I20250904 19:58:55.203697 10338 fs_manager.cc:362] Metadata directory not provided
I20250904 19:58:55.203933 10338 fs_manager.cc:365] Using existing metadata directory in first data directory
I20250904 19:58:55.214478 10338 fs_manager.cc:714] Time spent opening directory manager: real 0.009s user 0.008s sys 0.001s
I20250904 19:58:55.220726 17020 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 19:58:55.221786 10338 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20250904 19:58:55.222157 10338 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-1-root
uuid: "14c649e56e32428798b4895763179e23"
format_stamp: "Formatted at 2025-09-04 19:58:52 on dist-test-slave-f3xj"
I20250904 19:58:55.222484 10338 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-1-root
metadata directory: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-1-root
1 data directories: /tmp/dist-test-taskfEgWkn/test-tmp/ksck_remote-test.0.RemoteKsckTest.TestClusterWithLocation.1757015869512910-10338-0/minicluster-data/master-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 19:58:55.282598 17032 catalog_manager.cc:1261] Loaded cluster ID: 358caa29654e4b63b00181d648cf2e92
I20250904 19:58:55.282929 17032 catalog_manager.cc:1554] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3: loading cluster ID for follower catalog manager: success
I20250904 19:58:55.288069 10338 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 19:58:55.289456 10338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 19:58:55.291646 16962 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" instance_seqno: 1757015933304020) as {username='slave'} at 127.0.0.1:56592; Asking this server to re-register.
I20250904 19:58:55.291669 16963 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" instance_seqno: 1757015933104432) as {username='slave'} at 127.0.0.1:56604; Asking this server to re-register.
I20250904 19:58:55.293421 16961 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" instance_seqno: 1757015933517657) as {username='slave'} at 127.0.0.1:56614; Asking this server to re-register.
I20250904 19:58:55.293895 16790 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:55.293898 16698 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:55.393187 16868 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:55.563247 16961 ts_manager.cc:194] Registered new tserver with Master: 1121c8a9c5f64923856f52dd7c39b015 (127.10.24.130:41795)
I20250904 19:58:55.572674 16963 ts_manager.cc:194] Registered new tserver with Master: 34fcdc21eb7e477ebab2618592e9c276 (127.10.24.129:46843)
I20250904 19:58:55.691910 17032 catalog_manager.cc:1576] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3: acquiring CA information for follower catalog manager: success
I20250904 19:58:55.755594 17032 catalog_manager.cc:1604] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:58:55.884950 16962 ts_manager.cc:194] Registered new tserver with Master: f34422a0cbdc4389a55bd0e2b2cd3c51 (127.10.24.131:41725)
I20250904 19:58:55.983057 10338 rpc_server.cc:307] RPC server started. Bound to: 127.10.24.189:41319
I20250904 19:58:55.983213 17084 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.24.189:41319 every 8 connection(s)
I20250904 19:58:55.997820 10338 master.cc:561] Master@127.10.24.188:37777 shutting down...
I20250904 19:58:56.017093 17085 sys_catalog.cc:263] Verifying existing consensus state
I20250904 19:58:56.037703 16787 heartbeater.cc:344] Connected to a master server at 127.10.24.189:41319
I20250904 19:58:56.045028 17085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23: Bootstrap starting.
I20250904 19:58:56.088896 16697 heartbeater.cc:344] Connected to a master server at 127.10.24.189:41319
I20250904 19:58:56.094046 17090 raft_consensus.cc:491] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [term 1 FOLLOWER]: Starting pre-election (detected failure of leader de9343d4ee8749d5b98effa595ef18e3)
I20250904 19:58:56.094676 17090 raft_consensus.cc:513] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } }
I20250904 19:58:56.100144 17090 leader_election.cc:290] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers de9343d4ee8749d5b98effa595ef18e3 (127.10.24.190:43389), 14c649e56e32428798b4895763179e23 (127.10.24.189:41319)
I20250904 19:58:56.113838 10338 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 19:58:56.115942 16866 heartbeater.cc:344] Connected to a master server at 127.10.24.189:41319
I20250904 19:58:56.127174 10338 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 19:58:56.128559 10338 tablet_replica.cc:331] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca: stopping tablet replica
I20250904 19:58:56.153194 16973 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "de9343d4ee8749d5b98effa595ef18e3" is_pre_election: true
I20250904 19:58:56.154000 16973 raft_consensus.cc:2466] T 00000000000000000000000000000000 P de9343d4ee8749d5b98effa595ef18e3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3609a8f04c864dbeb9e91fff62ecc4ca in term 1.
W20250904 19:58:56.155952 16532 proxy.cc:239] Call had error, refreshing address and retrying: Service unavailable: reactor is shutting down (error 108) [suppressed 17 similar messages]
I20250904 19:58:56.158425 16866 heartbeater.cc:344] Connected to a master server at 127.10.24.189:41319
W20250904 19:58:56.165854 17095 leader_election.cc:336] T 00000000000000000000000000000000 P 3609a8f04c864dbeb9e91fff62ecc4ca [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 14c649e56e32428798b4895763179e23 (127.10.24.189:41319): Service unavailable: reactor is shutting down (error 108)
I20250904 19:58:56.166776 17085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=7 ignored=0} mutations{seen=6 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20250904 19:58:56.167425 16803 connection.cc:537] client connection to 127.10.24.188:37777 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20250904 19:58:56.175513 17085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23: Bootstrap complete.
I20250904 19:58:56.178064 17085 raft_consensus.cc:357] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } }
I20250904 19:58:56.178622 17085 raft_consensus.cc:738] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14c649e56e32428798b4895763179e23, State: Initialized, Role: FOLLOWER
F20250904 19:58:56.179804 10338 leader_election.cc:206] Check failed: has_responded_
*** Check failure stack trace: ***
I20250904 19:58:56.179286 17085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 1.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } }
*** Aborted at 1757015936 (unix time) try "date -d @1757015936" if you are using GNU date ***
I20250904 19:58:56.195019 17096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "de9343d4ee8749d5b98effa595ef18e3" member_type: VOTER last_known_addr { host: "127.10.24.190" port: 43389 } } peers { permanent_uuid: "14c649e56e32428798b4895763179e23" member_type: VOTER last_known_addr { host: "127.10.24.189" port: 41319 } } peers { permanent_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" member_type: VOTER last_known_addr { host: "127.10.24.188" port: 37777 } } }
I20250904 19:58:56.202303 17060 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3609a8f04c864dbeb9e91fff62ecc4ca" candidate_term: 2 candidate_status { last_received { term: 1 index: 9 } } ignore_live_leader: false dest_uuid: "14c649e56e32428798b4895763179e23" is_pre_election: true
I20250904 19:58:56.202217 17085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 19:58:56.202927 17096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [sys.catalog]: This master's current role is: FOLLOWER
I20250904 19:58:56.203181 17060 raft_consensus.cc:2466] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3609a8f04c864dbeb9e91fff62ecc4ca in term 1.
W20250904 19:58:56.204219 17026 connection.cc:441] server connection from 127.0.0.1:54148 torn down before Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54148 (request call id 0) could send its response
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e800002862) received by PID 10338 (TID 0x7f4d36f7c840) from PID 10338; stack trace: ***
@ 0x2d16e0 __tsan::CallUserSignalHandler() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
@ 0x2d3bc3 rtl_sigaction() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
I20250904 19:58:56.266144 17049 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "1121c8a9c5f64923856f52dd7c39b015" instance_seqno: 1757015933304020) as {username='slave'} at 127.0.0.1:54134; Asking this server to re-register.
I20250904 19:58:56.269023 16787 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:56.353358 17049 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "34fcdc21eb7e477ebab2618592e9c276" instance_seqno: 1757015933104432) as {username='slave'} at 127.0.0.1:54106; Asking this server to re-register.
@ 0x7f4d3e0b0980 (unknown) at ??:0
I20250904 19:58:56.380795 17108 catalog_manager.cc:1261] Loaded cluster ID: 358caa29654e4b63b00181d648cf2e92
I20250904 19:58:56.381536 17108 catalog_manager.cc:1554] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23: loading cluster ID for follower catalog manager: success
I20250904 19:58:56.390074 17048 master_service.cc:432] Got heartbeat from unknown tserver (permanent_uuid: "f34422a0cbdc4389a55bd0e2b2cd3c51" instance_seqno: 1757015933517657) as {username='slave'} at 127.0.0.1:54122; Asking this server to re-register.
I20250904 19:58:56.397461 16866 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:56.466128 17108 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23: acquiring CA information for follower catalog manager: success
I20250904 19:58:56.481422 16697 heartbeater.cc:461] Registering TS with master...
I20250904 19:58:56.482000 17050 ts_manager.cc:194] Registered new tserver with Master: 1121c8a9c5f64923856f52dd7c39b015 (127.10.24.130:41795)
I20250904 19:58:56.563040 17108 catalog_manager.cc:1604] T 00000000000000000000000000000000 P 14c649e56e32428798b4895763179e23: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20250904 19:58:56.645882 17047 ts_manager.cc:194] Registered new tserver with Master: f34422a0cbdc4389a55bd0e2b2cd3c51 (127.10.24.131:41725)
@ 0x7f4d37c3cfb7 gsignal at ??:0
W20250904 19:58:56.680317 16713 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.10.24.129:46843, user_credentials={real_user=slave}} blocked reactor thread for 301379us
I20250904 19:58:56.692951 17050 ts_manager.cc:194] Registered new tserver with Master: 34fcdc21eb7e477ebab2618592e9c276 (127.10.24.129:46843)
@ 0x7f4d37c3e921 abort at ??:0
@ 0x2d76d6 abort at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
@ 0x7f4d3a5b8da4 google::LogMessage::SendToLog() at ??:0
@ 0x7f4d3a5b9950 google::LogMessage::Flush() at ??:0
@ 0x7f4d3a5bea8b google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f4d3da4832c kudu::consensus::LeaderElection::~LeaderElection() at ??:0
@ 0x7f4d3da55b6f kudu::RefCountedThreadSafe<>::DeleteInternal() at ??:0
@ 0x7f4d3da55b3a kudu::DefaultRefCountedThreadSafeTraits<>::Destruct() at ??:0
@ 0x7f4d3da55af6 kudu::RefCountedThreadSafe<>::Release() at ??:0
@ 0x7f4d3da4d5c7 scoped_refptr<>::~scoped_refptr() at ??:0
@ 0x7f4d3da49963 kudu::consensus::LeaderElection::Run()::$_0::~Run() at ??:0
@ 0x7f4d3da4b54a std::__1::__compressed_pair_elem<>::~__compressed_pair_elem() at ??:0
@ 0x7f4d3da4be6a std::__1::__function::__alloc_func<>::destroy() at ??:0
@ 0x7f4d3da4b13b std::__1::__function::__func<>::destroy_deallocate() at ??:0
@ 0x3760db std::__1::__function::__value_func<>::~__value_func() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:?
@ 0x36827a std::__1::function<>::~function() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2543
@ 0x7f4d3bcf519e kudu::rpc::Proxy::RefreshDnsAndEnqueueRequest()::$_0::~RefreshDnsAndEnqueueRequest() at ??:0
@ 0x7f4d3bcf655a std::__1::__compressed_pair_elem<>::~__compressed_pair_elem() at ??:0
@ 0x7f4d3bcf6e8a std::__1::__function::__alloc_func<>::destroy() at ??:0
@ 0x7f4d3bcf612b std::__1::__function::__func<>::destroy_deallocate() at ??:0
@ 0x7f4d43c5077b std::__1::__function::__value_func<>::~__value_func() at ??:0
@ 0x7f4d43c2d2ca std::__1::function<>::~function() at ??:0
@ 0x7f4d3aea52ab kudu::DnsResolver::RefreshAddressesAsync()::$_1::~RefreshAddressesAsync() at ??:0
@ 0x7f4d3aea74aa std::__1::__compressed_pair_elem<>::~__compressed_pair_elem() at ??:0
@ 0x7f4d3aea7e1a std::__1::__function::__alloc_func<>::destroy() at ??:0
@ 0x7f4d3aea704b std::__1::__function::__func<>::destroy_deallocate() at ??:0
@ 0x3760db std::__1::__function::__value_func<>::~__value_func() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:?
@ 0x36827a std::__1::function<>::~function() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2543