[==========] Running 18 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 8 tests from TServerQuiescingITest
[ RUN ] TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20250904 13:57:21.915227 10634 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.98.190:34657
I20250904 13:57:21.917083 10634 env_posix.cc:2264] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250904 13:57:21.918354 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:21.937321 10641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:21.937409 10640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:21.937390 10643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:23.151852 10642 instance_detector.cc:116] could not retrieve GCE instance metadata: Timed out: curl timeout: Timeout was reached: Resolving timed out after 1212 milliseconds
I20250904 13:57:23.151994 10634 server_base.cc:1042] Not found: could not retrieve instance metadata: unable to detect cloud type of this node, probably running in non-cloud environment
I20250904 13:57:23.153393 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:23.153642 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:23.153841 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994243153816 us; error 0 us; skew 500 ppm
I20250904 13:57:23.154743 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:23.161530 10634 webserver.cc:480] Webserver started at http://127.10.98.190:45679/ using document root <none> and password file <none>
I20250904 13:57:23.162555 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:23.162786 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:23.163285 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:23.169642 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections.1756994241738102-10634-0/minicluster-data/master-0-root/instance:
uuid: "53f6277f96774ff8a2fbe3258bc0c659"
format_stamp: "Formatted at 2025-09-04 13:57:23 on dist-test-slave-jx76"
I20250904 13:57:23.178464 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.008s user 0.006s sys 0.001s
I20250904 13:57:23.184162 10649 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:23.185460 10634 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.005s sys 0.001s
I20250904 13:57:23.185820 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections.1756994241738102-10634-0/minicluster-data/master-0-root
uuid: "53f6277f96774ff8a2fbe3258bc0c659"
format_stamp: "Formatted at 2025-09-04 13:57:23 on dist-test-slave-jx76"
I20250904 13:57:23.186237 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections.1756994241738102-10634-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections.1756994241738102-10634-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections.1756994241738102-10634-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:23.236397 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:23.238050 10634 env_posix.cc:2264] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250904 13:57:23.238732 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:23.317231 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.190:34657
I20250904 13:57:23.317350 10700 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.190:34657 every 8 connection(s)
I20250904 13:57:23.325282 10701 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:23.349592 10701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659: Bootstrap starting.
I20250904 13:57:23.355872 10701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:23.357877 10701 log.cc:826] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659: Log is configured to *not* fsync() on all Append() calls
I20250904 13:57:23.363204 10701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659: No bootstrap required, opened a new log
I20250904 13:57:23.383778 10701 raft_consensus.cc:357] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f6277f96774ff8a2fbe3258bc0c659" member_type: VOTER }
I20250904 13:57:23.384385 10701 raft_consensus.cc:383] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:23.384680 10701 raft_consensus.cc:738] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f6277f96774ff8a2fbe3258bc0c659, State: Initialized, Role: FOLLOWER
I20250904 13:57:23.385568 10701 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f6277f96774ff8a2fbe3258bc0c659" member_type: VOTER }
I20250904 13:57:23.386080 10701 raft_consensus.cc:397] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:57:23.386409 10701 raft_consensus.cc:491] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250904 13:57:23.386763 10701 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:23.391024 10701 raft_consensus.cc:513] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f6277f96774ff8a2fbe3258bc0c659" member_type: VOTER }
I20250904 13:57:23.391822 10701 leader_election.cc:304] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53f6277f96774ff8a2fbe3258bc0c659; no voters:
I20250904 13:57:23.393607 10701 leader_election.cc:290] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:57:23.393958 10704 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:23.396955 10704 raft_consensus.cc:695] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659 [term 1 LEADER]: Becoming Leader. State: Replica: 53f6277f96774ff8a2fbe3258bc0c659, State: Running, Role: LEADER
I20250904 13:57:23.397892 10704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f6277f96774ff8a2fbe3258bc0c659" member_type: VOTER }
I20250904 13:57:23.399142 10701 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 13:57:23.410530 10705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53f6277f96774ff8a2fbe3258bc0c659. Latest consensus state: current_term: 1 leader_uuid: "53f6277f96774ff8a2fbe3258bc0c659" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f6277f96774ff8a2fbe3258bc0c659" member_type: VOTER } }
I20250904 13:57:23.411582 10705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659 [sys.catalog]: This master's current role is: LEADER
I20250904 13:57:23.411980 10706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53f6277f96774ff8a2fbe3258bc0c659" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53f6277f96774ff8a2fbe3258bc0c659" member_type: VOTER } }
I20250904 13:57:23.412775 10706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659 [sys.catalog]: This master's current role is: LEADER
I20250904 13:57:23.415449 10714 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 13:57:23.429665 10714 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 13:57:23.431880 10634 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 13:57:23.448376 10714 catalog_manager.cc:1349] Generated new cluster ID: 72f0806437be4f7da43c571ea877e6a5
I20250904 13:57:23.448729 10714 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 13:57:23.479669 10714 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 13:57:23.481364 10714 catalog_manager.cc:1506] Loading token signing keys...
I20250904 13:57:23.495824 10714 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659: Generated new TSK 0
I20250904 13:57:23.497171 10714 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 13:57:23.563563 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:23.572690 10723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:23.573947 10724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:23.579605 10634 server_base.cc:1047] running on GCE node
W20250904 13:57:23.580075 10726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:23.581130 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:23.581388 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:23.581576 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994243581554 us; error 0 us; skew 500 ppm
I20250904 13:57:23.582170 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:23.586889 10634 webserver.cc:480] Webserver started at http://127.10.98.129:44283/ using document root <none> and password file <none>
I20250904 13:57:23.587502 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:23.587714 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:23.588006 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:23.589227 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections.1756994241738102-10634-0/minicluster-data/ts-0-root/instance:
uuid: "431fe14e8b31422ba906a93b6e5f45be"
format_stamp: "Formatted at 2025-09-04 13:57:23 on dist-test-slave-jx76"
I20250904 13:57:23.595511 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20250904 13:57:23.600417 10731 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:23.601329 10634 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20250904 13:57:23.601704 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections.1756994241738102-10634-0/minicluster-data/ts-0-root
uuid: "431fe14e8b31422ba906a93b6e5f45be"
format_stamp: "Formatted at 2025-09-04 13:57:23 on dist-test-slave-jx76"
I20250904 13:57:23.602114 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections.1756994241738102-10634-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections.1756994241738102-10634-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections.1756994241738102-10634-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:23.628444 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:23.630461 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:23.633430 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:57:23.639227 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:57:23.639585 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:23.639894 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:57:23.640165 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:23.741364 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.129:42753
I20250904 13:57:23.741446 10793 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.129:42753 every 8 connection(s)
I20250904 13:57:23.749279 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:23.759613 10798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:23.761382 10799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:23.771766 10634 server_base.cc:1047] running on GCE node
W20250904 13:57:23.772374 10801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:23.773648 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:23.773940 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:23.775126 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994243775096 us; error 0 us; skew 500 ppm
I20250904 13:57:23.775840 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:23.775801 10794 heartbeater.cc:344] Connected to a master server at 127.10.98.190:34657
I20250904 13:57:23.776479 10794 heartbeater.cc:461] Registering TS with master...
I20250904 13:57:23.777734 10794 heartbeater.cc:507] Master 127.10.98.190:34657 requested a full tablet report, sending...
I20250904 13:57:23.779371 10634 webserver.cc:480] Webserver started at http://127.10.98.130:45595/ using document root <none> and password file <none>
I20250904 13:57:23.780123 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:23.780392 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:23.780761 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:23.781165 10666 ts_manager.cc:194] Registered new tserver with Master: 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753)
I20250904 13:57:23.782357 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections.1756994241738102-10634-0/minicluster-data/ts-1-root/instance:
uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c"
format_stamp: "Formatted at 2025-09-04 13:57:23 on dist-test-slave-jx76"
I20250904 13:57:23.783569 10666 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47434
I20250904 13:57:23.789611 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20250904 13:57:23.795472 10806 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:23.796375 10634 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20250904 13:57:23.796732 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections.1756994241738102-10634-0/minicluster-data/ts-1-root
uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c"
format_stamp: "Formatted at 2025-09-04 13:57:23 on dist-test-slave-jx76"
I20250904 13:57:23.797113 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections.1756994241738102-10634-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections.1756994241738102-10634-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections.1756994241738102-10634-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:23.832336 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:23.833727 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:23.836561 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:57:23.840230 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:57:23.840440 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:23.840716 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:57:23.840865 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:23.945936 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.130:39863
I20250904 13:57:23.946280 10868 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.130:39863 every 8 connection(s)
I20250904 13:57:23.954320 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:23.966251 10872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:23.968654 10869 heartbeater.cc:344] Connected to a master server at 127.10.98.190:34657
I20250904 13:57:23.969137 10869 heartbeater.cc:461] Registering TS with master...
W20250904 13:57:23.969995 10873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:23.970286 10869 heartbeater.cc:507] Master 127.10.98.190:34657 requested a full tablet report, sending...
W20250904 13:57:23.971973 10875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:23.973021 10666 ts_manager.cc:194] Registered new tserver with Master: 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:23.974026 10634 server_base.cc:1047] running on GCE node
I20250904 13:57:23.974975 10666 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47446
I20250904 13:57:23.975169 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:23.975442 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:23.975644 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994243975625 us; error 0 us; skew 500 ppm
I20250904 13:57:23.976363 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:23.979351 10634 webserver.cc:480] Webserver started at http://127.10.98.131:40523/ using document root <none> and password file <none>
I20250904 13:57:23.979889 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:23.980095 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:23.980369 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:23.981500 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections.1756994241738102-10634-0/minicluster-data/ts-2-root/instance:
uuid: "926518f7b9994fc288cbdff9d74323a3"
format_stamp: "Formatted at 2025-09-04 13:57:23 on dist-test-slave-jx76"
I20250904 13:57:23.986698 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20250904 13:57:23.990439 10880 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:23.991361 10634 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20250904 13:57:23.991770 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections.1756994241738102-10634-0/minicluster-data/ts-2-root
uuid: "926518f7b9994fc288cbdff9d74323a3"
format_stamp: "Formatted at 2025-09-04 13:57:23 on dist-test-slave-jx76"
I20250904 13:57:23.992203 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections.1756994241738102-10634-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections.1756994241738102-10634-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections.1756994241738102-10634-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:24.030619 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:24.032461 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:24.034366 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:57:24.037889 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:57:24.038225 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20250904 13:57:24.038544 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:57:24.038770 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:24.134450 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.131:32881
I20250904 13:57:24.134516 10942 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.131:32881 every 8 connection(s)
I20250904 13:57:24.156113 10943 heartbeater.cc:344] Connected to a master server at 127.10.98.190:34657
I20250904 13:57:24.156533 10943 heartbeater.cc:461] Registering TS with master...
I20250904 13:57:24.157485 10943 heartbeater.cc:507] Master 127.10.98.190:34657 requested a full tablet report, sending...
I20250904 13:57:24.159829 10666 ts_manager.cc:194] Registered new tserver with Master: 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:24.160313 10634 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018478511s
I20250904 13:57:24.161350 10666 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47448
I20250904 13:57:24.211139 10634 test_util.cc:276] Using random seed: 432828835
I20250904 13:57:24.263618 10666 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47470:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 13:57:24.267894 10666 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 13:57:24.383790 10908 tablet_service.cc:1468] Processing CreateTablet for tablet 8718fc5fdf154903a9039ed2c48a85b7 (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION VALUES < 214748364
I20250904 13:57:24.391213 10908 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8718fc5fdf154903a9039ed2c48a85b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.386142 10907 tablet_service.cc:1468] Processing CreateTablet for tablet 7cc86826b94e49289260ba78159669db (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20250904 13:57:24.394541 10902 tablet_service.cc:1468] Processing CreateTablet for tablet 1fd1a3b7dd454961b9ed1dbc3e0af7ef (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20250904 13:57:24.395460 10907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cc86826b94e49289260ba78159669db. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.386858 10905 tablet_service.cc:1468] Processing CreateTablet for tablet 5c80f91315f84cf49f62445010a99d83 (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20250904 13:57:24.396642 10905 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c80f91315f84cf49f62445010a99d83. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.389328 10904 tablet_service.cc:1468] Processing CreateTablet for tablet 8f248691b38b4c76add5969ac40fbdc0 (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20250904 13:57:24.399924 10904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f248691b38b4c76add5969ac40fbdc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.384310 10906 tablet_service.cc:1468] Processing CreateTablet for tablet 6f08a2bf1d1841a19924894a1af8dcac (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20250904 13:57:24.391851 10903 tablet_service.cc:1468] Processing CreateTablet for tablet 27b15f5a5a6f481cbb1af6df1053357a (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20250904 13:57:24.402204 10906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f08a2bf1d1841a19924894a1af8dcac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.403405 10903 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27b15f5a5a6f481cbb1af6df1053357a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.405761 10902 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fd1a3b7dd454961b9ed1dbc3e0af7ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.414047 10901 tablet_service.cc:1468] Processing CreateTablet for tablet a1cd635db3064f9c94a5dbc6d620db61 (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20250904 13:57:24.438400 10900 tablet_service.cc:1468] Processing CreateTablet for tablet a1d2da9bd6e0405fa033babd924eb743 (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20250904 13:57:24.441166 10901 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1cd635db3064f9c94a5dbc6d620db61. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.438532 10832 tablet_service.cc:1468] Processing CreateTablet for tablet 6f08a2bf1d1841a19924894a1af8dcac (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20250904 13:57:24.439227 10759 tablet_service.cc:1468] Processing CreateTablet for tablet 8718fc5fdf154903a9039ed2c48a85b7 (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION VALUES < 214748364
I20250904 13:57:24.448407 10833 tablet_service.cc:1468] Processing CreateTablet for tablet 7cc86826b94e49289260ba78159669db (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20250904 13:57:24.453749 10899 tablet_service.cc:1468] Processing CreateTablet for tablet bb8182b451f946b78811668ce1d0ce37 (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20250904 13:57:24.448377 10758 tablet_service.cc:1468] Processing CreateTablet for tablet 7cc86826b94e49289260ba78159669db (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20250904 13:57:24.449868 10757 tablet_service.cc:1468] Processing CreateTablet for tablet 6f08a2bf1d1841a19924894a1af8dcac (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20250904 13:57:24.453680 10828 tablet_service.cc:1468] Processing CreateTablet for tablet 1fd1a3b7dd454961b9ed1dbc3e0af7ef (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20250904 13:57:24.457940 10900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1d2da9bd6e0405fa033babd924eb743. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.462874 10833 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cc86826b94e49289260ba78159669db. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.450678 10830 tablet_service.cc:1468] Processing CreateTablet for tablet 8f248691b38b4c76add5969ac40fbdc0 (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20250904 13:57:24.467366 10831 tablet_service.cc:1468] Processing CreateTablet for tablet 5c80f91315f84cf49f62445010a99d83 (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20250904 13:57:24.471411 10834 tablet_service.cc:1468] Processing CreateTablet for tablet 8718fc5fdf154903a9039ed2c48a85b7 (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION VALUES < 214748364
I20250904 13:57:24.471117 10757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f08a2bf1d1841a19924894a1af8dcac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.464936 10829 tablet_service.cc:1468] Processing CreateTablet for tablet 27b15f5a5a6f481cbb1af6df1053357a (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20250904 13:57:24.464555 10756 tablet_service.cc:1468] Processing CreateTablet for tablet 5c80f91315f84cf49f62445010a99d83 (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20250904 13:57:24.464689 10754 tablet_service.cc:1468] Processing CreateTablet for tablet 27b15f5a5a6f481cbb1af6df1053357a (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20250904 13:57:24.483875 10832 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f08a2bf1d1841a19924894a1af8dcac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.467460 10755 tablet_service.cc:1468] Processing CreateTablet for tablet 8f248691b38b4c76add5969ac40fbdc0 (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20250904 13:57:24.489320 10899 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb8182b451f946b78811668ce1d0ce37. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.491698 10828 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fd1a3b7dd454961b9ed1dbc3e0af7ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.492899 10758 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cc86826b94e49289260ba78159669db. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.491652 10825 tablet_service.cc:1468] Processing CreateTablet for tablet bb8182b451f946b78811668ce1d0ce37 (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20250904 13:57:24.494506 10753 tablet_service.cc:1468] Processing CreateTablet for tablet 1fd1a3b7dd454961b9ed1dbc3e0af7ef (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20250904 13:57:24.490005 10827 tablet_service.cc:1468] Processing CreateTablet for tablet a1cd635db3064f9c94a5dbc6d620db61 (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20250904 13:57:24.491608 10826 tablet_service.cc:1468] Processing CreateTablet for tablet a1d2da9bd6e0405fa033babd924eb743 (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20250904 13:57:24.491576 10752 tablet_service.cc:1468] Processing CreateTablet for tablet a1cd635db3064f9c94a5dbc6d620db61 (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20250904 13:57:24.498833 10751 tablet_service.cc:1468] Processing CreateTablet for tablet a1d2da9bd6e0405fa033babd924eb743 (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20250904 13:57:24.513464 10829 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27b15f5a5a6f481cbb1af6df1053357a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.516633 10826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1d2da9bd6e0405fa033babd924eb743. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.511482 10754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27b15f5a5a6f481cbb1af6df1053357a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.511065 10750 tablet_service.cc:1468] Processing CreateTablet for tablet bb8182b451f946b78811668ce1d0ce37 (DEFAULT_TABLE table=test-workload [id=571a597ed7f54e6485b337150aaabb48]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20250904 13:57:24.520114 10827 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1cd635db3064f9c94a5dbc6d620db61. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.546788 10753 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1fd1a3b7dd454961b9ed1dbc3e0af7ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.548944 10830 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f248691b38b4c76add5969ac40fbdc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.550758 10755 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f248691b38b4c76add5969ac40fbdc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.556279 10752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1cd635db3064f9c94a5dbc6d620db61. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.566390 10831 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c80f91315f84cf49f62445010a99d83. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.581339 10756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c80f91315f84cf49f62445010a99d83. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.589937 10825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb8182b451f946b78811668ce1d0ce37. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.594237 10834 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8718fc5fdf154903a9039ed2c48a85b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.596659 10751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1d2da9bd6e0405fa033babd924eb743. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.620790 10759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8718fc5fdf154903a9039ed2c48a85b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.648464 10750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb8182b451f946b78811668ce1d0ce37. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:24.787030 10794 heartbeater.cc:499] Master 127.10.98.190:34657 was elected leader, sending a full tablet report...
I20250904 13:57:24.810796 10966 tablet_bootstrap.cc:492] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3: Bootstrap starting.
I20250904 13:57:24.835320 10967 tablet_bootstrap.cc:492] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Bootstrap starting.
I20250904 13:57:24.876065 10967 tablet_bootstrap.cc:654] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:24.878784 10966 tablet_bootstrap.cc:654] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:24.901893 10967 tablet_bootstrap.cc:492] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c: No bootstrap required, opened a new log
I20250904 13:57:24.902473 10967 ts_tablet_manager.cc:1397] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Time spent bootstrapping tablet: real 0.068s user 0.010s sys 0.002s
I20250904 13:57:24.906576 10968 tablet_bootstrap.cc:492] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be: Bootstrap starting.
I20250904 13:57:24.914371 10967 raft_consensus.cc:357] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:24.916018 10967 raft_consensus.cc:383] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:24.916323 10967 raft_consensus.cc:738] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Initialized, Role: FOLLOWER
I20250904 13:57:24.916926 10967 consensus_queue.cc:260] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:24.919651 10966 tablet_bootstrap.cc:492] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3: No bootstrap required, opened a new log
I20250904 13:57:24.920161 10966 ts_tablet_manager.cc:1397] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3: Time spent bootstrapping tablet: real 0.110s user 0.008s sys 0.014s
I20250904 13:57:24.920948 10968 tablet_bootstrap.cc:654] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:24.923324 10869 heartbeater.cc:499] Master 127.10.98.190:34657 was elected leader, sending a full tablet report...
I20250904 13:57:24.923460 10967 ts_tablet_manager.cc:1428] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Time spent starting tablet: real 0.021s user 0.000s sys 0.008s
I20250904 13:57:24.923499 10966 raft_consensus.cc:357] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } }
I20250904 13:57:24.924597 10966 raft_consensus.cc:383] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:24.924952 10967 tablet_bootstrap.cc:492] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Bootstrap starting.
I20250904 13:57:24.924901 10966 raft_consensus.cc:738] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 926518f7b9994fc288cbdff9d74323a3, State: Initialized, Role: FOLLOWER
I20250904 13:57:24.925855 10966 consensus_queue.cc:260] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } }
I20250904 13:57:24.932255 10967 tablet_bootstrap.cc:654] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:24.950657 10966 ts_tablet_manager.cc:1428] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3: Time spent starting tablet: real 0.030s user 0.009s sys 0.002s
I20250904 13:57:24.951690 10943 heartbeater.cc:499] Master 127.10.98.190:34657 was elected leader, sending a full tablet report...
W20250904 13:57:24.957238 10870 tablet.cc:2378] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250904 13:57:24.960919 10966 tablet_bootstrap.cc:492] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3: Bootstrap starting.
I20250904 13:57:24.969827 10968 tablet_bootstrap.cc:492] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be: No bootstrap required, opened a new log
I20250904 13:57:24.970381 10968 ts_tablet_manager.cc:1397] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be: Time spent bootstrapping tablet: real 0.087s user 0.009s sys 0.042s
I20250904 13:57:24.973169 10968 raft_consensus.cc:357] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:24.973904 10968 raft_consensus.cc:383] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:24.974217 10968 raft_consensus.cc:738] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 431fe14e8b31422ba906a93b6e5f45be, State: Initialized, Role: FOLLOWER
I20250904 13:57:24.974426 10973 raft_consensus.cc:491] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:24.975064 10968 consensus_queue.cc:260] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:24.975236 10973 raft_consensus.cc:513] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } }
I20250904 13:57:24.984786 10968 ts_tablet_manager.cc:1428] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be: Time spent starting tablet: real 0.014s user 0.010s sys 0.002s
I20250904 13:57:24.985800 10968 tablet_bootstrap.cc:492] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be: Bootstrap starting.
I20250904 13:57:24.991873 10968 tablet_bootstrap.cc:654] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.003557 10968 tablet_bootstrap.cc:492] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be: No bootstrap required, opened a new log
I20250904 13:57:25.004436 10968 ts_tablet_manager.cc:1397] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be: Time spent bootstrapping tablet: real 0.019s user 0.015s sys 0.003s
I20250904 13:57:25.007267 10967 tablet_bootstrap.cc:492] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c: No bootstrap required, opened a new log
I20250904 13:57:25.007856 10967 ts_tablet_manager.cc:1397] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Time spent bootstrapping tablet: real 0.083s user 0.049s sys 0.030s
I20250904 13:57:25.007783 10968 raft_consensus.cc:357] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.008590 10968 raft_consensus.cc:383] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.008890 10968 raft_consensus.cc:738] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 431fe14e8b31422ba906a93b6e5f45be, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.009575 10968 consensus_queue.cc:260] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.012027 10968 ts_tablet_manager.cc:1428] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20250904 13:57:25.013006 10968 tablet_bootstrap.cc:492] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be: Bootstrap starting.
I20250904 13:57:25.016037 10973 leader_election.cc:290] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753)
I20250904 13:57:25.018421 10967 raft_consensus.cc:357] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.019434 10967 raft_consensus.cc:383] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.019675 10968 tablet_bootstrap.cc:654] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.019883 10967 raft_consensus.cc:738] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.010077 10966 tablet_bootstrap.cc:654] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.026799 10967 consensus_queue.cc:260] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.038429 10967 ts_tablet_manager.cc:1428] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Time spent starting tablet: real 0.030s user 0.007s sys 0.001s
I20250904 13:57:25.039544 10967 tablet_bootstrap.cc:492] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Bootstrap starting.
I20250904 13:57:25.053752 10967 tablet_bootstrap.cc:654] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.061442 10966 tablet_bootstrap.cc:492] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3: No bootstrap required, opened a new log
I20250904 13:57:25.061967 10966 ts_tablet_manager.cc:1397] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3: Time spent bootstrapping tablet: real 0.101s user 0.014s sys 0.010s
I20250904 13:57:25.064477 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8718fc5fdf154903a9039ed2c48a85b7" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:25.065733 10966 raft_consensus.cc:357] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.066519 10966 raft_consensus.cc:383] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.066808 10966 raft_consensus.cc:738] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 926518f7b9994fc288cbdff9d74323a3, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.067485 10966 consensus_queue.cc:260] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.074191 10966 ts_tablet_manager.cc:1428] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20250904 13:57:25.075157 10966 tablet_bootstrap.cc:492] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3: Bootstrap starting.
I20250904 13:57:25.080276 10967 tablet_bootstrap.cc:492] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: No bootstrap required, opened a new log
I20250904 13:57:25.080717 10971 raft_consensus.cc:491] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:25.080796 10967 ts_tablet_manager.cc:1397] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Time spent bootstrapping tablet: real 0.041s user 0.011s sys 0.018s
W20250904 13:57:25.081070 10884 leader_election.cc:343] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.081286 10971 raft_consensus.cc:513] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.083998 10971 leader_election.cc:290] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:25.084213 10967 raft_consensus.cc:357] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.085132 10967 raft_consensus.cc:383] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.085522 10967 raft_consensus.cc:738] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.086712 10968 tablet_bootstrap.cc:492] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be: No bootstrap required, opened a new log
I20250904 13:57:25.087159 10968 ts_tablet_manager.cc:1397] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be: Time spent bootstrapping tablet: real 0.074s user 0.006s sys 0.011s
I20250904 13:57:25.089721 10968 raft_consensus.cc:357] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.090498 10968 raft_consensus.cc:383] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.090816 10968 raft_consensus.cc:738] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 431fe14e8b31422ba906a93b6e5f45be, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.090696 10967 consensus_queue.cc:260] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.093071 10966 tablet_bootstrap.cc:654] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.091682 10968 consensus_queue.cc:260] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.094400 10971 raft_consensus.cc:491] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:25.094873 10971 raft_consensus.cc:513] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.096071 10968 ts_tablet_manager.cc:1428] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be: Time spent starting tablet: real 0.009s user 0.006s sys 0.001s
I20250904 13:57:25.096837 10971 leader_election.cc:290] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:25.097993 10968 tablet_bootstrap.cc:492] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be: Bootstrap starting.
I20250904 13:57:25.100113 10967 ts_tablet_manager.cc:1428] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Time spent starting tablet: real 0.019s user 0.006s sys 0.000s
I20250904 13:57:25.101047 10975 raft_consensus.cc:491] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:25.101727 10975 raft_consensus.cc:513] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.103340 10967 tablet_bootstrap.cc:492] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Bootstrap starting.
I20250904 13:57:25.104317 10975 leader_election.cc:290] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:25.111430 10967 tablet_bootstrap.cc:654] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.123802 10975 raft_consensus.cc:491] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:25.124522 10975 raft_consensus.cc:513] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.123415 10968 tablet_bootstrap.cc:654] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.131896 10975 leader_election.cc:290] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:25.134167 10966 tablet_bootstrap.cc:492] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3: No bootstrap required, opened a new log
I20250904 13:57:25.134686 10966 ts_tablet_manager.cc:1397] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3: Time spent bootstrapping tablet: real 0.060s user 0.012s sys 0.001s
I20250904 13:57:25.141827 10966 raft_consensus.cc:357] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.144043 10966 raft_consensus.cc:383] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.147338 10966 raft_consensus.cc:738] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 926518f7b9994fc288cbdff9d74323a3, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.155704 10769 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "27b15f5a5a6f481cbb1af6df1053357a" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
W20250904 13:57:25.159143 10810 leader_election.cc:343] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.156483 10966 consensus_queue.cc:260] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.160203 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1fd1a3b7dd454961b9ed1dbc3e0af7ef" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:25.161321 10768 raft_consensus.cc:2466] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 0.
I20250904 13:57:25.162881 10966 ts_tablet_manager.cc:1428] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3: Time spent starting tablet: real 0.028s user 0.008s sys 0.000s
I20250904 13:57:25.164450 10966 tablet_bootstrap.cc:492] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3: Bootstrap starting.
I20250904 13:57:25.164345 10810 leader_election.cc:304] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c; no voters:
I20250904 13:57:25.165874 10986 raft_consensus.cc:2802] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:57:25.166383 10986 raft_consensus.cc:491] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:57:25.166746 10986 raft_consensus.cc:3058] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.156518 10967 tablet_bootstrap.cc:492] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c: No bootstrap required, opened a new log
I20250904 13:57:25.174867 10967 ts_tablet_manager.cc:1397] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Time spent bootstrapping tablet: real 0.072s user 0.021s sys 0.011s
I20250904 13:57:25.175571 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1fd1a3b7dd454961b9ed1dbc3e0af7ef" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:25.179234 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1d2da9bd6e0405fa033babd924eb743" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:25.193909 10967 raft_consensus.cc:357] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
W20250904 13:57:25.195734 10735 leader_election.cc:343] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.195400 10967 raft_consensus.cc:383] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.198653 10967 raft_consensus.cc:738] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.199895 10967 consensus_queue.cc:260] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.201222 10968 tablet_bootstrap.cc:492] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be: No bootstrap required, opened a new log
I20250904 13:57:25.202677 10968 ts_tablet_manager.cc:1397] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be: Time spent bootstrapping tablet: real 0.105s user 0.021s sys 0.019s
I20250904 13:57:25.206832 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8718fc5fdf154903a9039ed2c48a85b7" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:25.208444 10973 raft_consensus.cc:491] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:25.208961 10973 raft_consensus.cc:513] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.209959 10968 raft_consensus.cc:357] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.210553 10966 tablet_bootstrap.cc:654] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.210726 10968 raft_consensus.cc:383] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.210980 10973 leader_election.cc:290] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:25.211086 10968 raft_consensus.cc:738] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 431fe14e8b31422ba906a93b6e5f45be, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.215148 10968 consensus_queue.cc:260] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.217749 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5c80f91315f84cf49f62445010a99d83" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:25.206039 10986 raft_consensus.cc:513] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.219044 10917 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1d2da9bd6e0405fa033babd924eb743" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
W20250904 13:57:25.220621 10884 leader_election.cc:343] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.219944 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5c80f91315f84cf49f62445010a99d83" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:25.222524 10918 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1fd1a3b7dd454961b9ed1dbc3e0af7ef" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
W20250904 13:57:25.222560 10735 leader_election.cc:343] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.222277 10884 leader_election.cc:304] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:25.226102 10973 raft_consensus.cc:2747] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250904 13:57:25.226189 10975 raft_consensus.cc:491] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:25.226734 10975 raft_consensus.cc:513] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
W20250904 13:57:25.227820 10884 leader_election.cc:343] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.229027 10975 leader_election.cc:290] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:25.229218 10973 raft_consensus.cc:491] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20250904 13:57:25.229343 10884 leader_election.cc:343] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.225035 10844 raft_consensus.cc:2391] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 431fe14e8b31422ba906a93b6e5f45be in current term 1: Already voted for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in this term.
I20250904 13:57:25.229700 10973 raft_consensus.cc:513] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.229902 10884 leader_election.cc:304] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:25.232640 10968 ts_tablet_manager.cc:1428] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be: Time spent starting tablet: real 0.029s user 0.009s sys 0.000s
I20250904 13:57:25.225152 10735 leader_election.cc:304] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:25.234932 10967 ts_tablet_manager.cc:1428] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Time spent starting tablet: real 0.060s user 0.001s sys 0.010s
I20250904 13:57:25.235466 10973 leader_election.cc:290] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
W20250904 13:57:25.236092 10735 leader_election.cc:343] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.236212 10973 raft_consensus.cc:2747] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250904 13:57:25.237525 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8f248691b38b4c76add5969ac40fbdc0" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:25.238382 10968 tablet_bootstrap.cc:492] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be: Bootstrap starting.
I20250904 13:57:25.238520 10918 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8f248691b38b4c76add5969ac40fbdc0" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:25.239771 10975 raft_consensus.cc:2747] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250904 13:57:25.240365 10967 tablet_bootstrap.cc:492] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Bootstrap starting.
W20250904 13:57:25.242684 10735 leader_election.cc:343] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881): Illegal state: must be running to vote when last-logged opid is not known
W20250904 13:57:25.243436 10735 leader_election.cc:343] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.243891 10735 leader_election.cc:304] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:25.244248 10968 tablet_bootstrap.cc:654] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.245162 10975 raft_consensus.cc:2747] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250904 13:57:25.246548 10966 tablet_bootstrap.cc:492] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3: No bootstrap required, opened a new log
I20250904 13:57:25.247193 10966 ts_tablet_manager.cc:1397] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3: Time spent bootstrapping tablet: real 0.083s user 0.013s sys 0.010s
I20250904 13:57:25.256757 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1fd1a3b7dd454961b9ed1dbc3e0af7ef" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be"
I20250904 13:57:25.258402 10768 raft_consensus.cc:3058] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.257817 10986 leader_election.cc:290] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Requested vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:25.257130 10966 raft_consensus.cc:357] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.260586 10966 raft_consensus.cc:383] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.261018 10966 raft_consensus.cc:738] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 926518f7b9994fc288cbdff9d74323a3, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.263831 10769 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7cc86826b94e49289260ba78159669db" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:25.263902 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7cc86826b94e49289260ba78159669db" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:25.264235 10967 tablet_bootstrap.cc:654] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.261766 10966 consensus_queue.cc:260] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.269202 10968 tablet_bootstrap.cc:492] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be: No bootstrap required, opened a new log
I20250904 13:57:25.269325 10966 ts_tablet_manager.cc:1428] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3: Time spent starting tablet: real 0.022s user 0.006s sys 0.002s
I20250904 13:57:25.269780 10968 ts_tablet_manager.cc:1397] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be: Time spent bootstrapping tablet: real 0.032s user 0.011s sys 0.014s
W20250904 13:57:25.267046 10884 leader_election.cc:343] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.270833 10966 tablet_bootstrap.cc:492] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3: Bootstrap starting.
W20250904 13:57:25.271970 10884 leader_election.cc:343] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.273072 10884 leader_election.cc:304] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:25.272233 10768 raft_consensus.cc:2466] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 1.
I20250904 13:57:25.274750 10986 raft_consensus.cc:491] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:25.275053 10996 raft_consensus.cc:2747] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250904 13:57:25.275466 10810 leader_election.cc:304] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c; no voters:
I20250904 13:57:25.276304 10986 raft_consensus.cc:513] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.277596 10971 raft_consensus.cc:2802] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:25.278594 10918 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "27b15f5a5a6f481cbb1af6df1053357a" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:25.282876 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
W20250904 13:57:25.287727 10810 leader_election.cc:343] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.288668 10986 leader_election.cc:290] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:25.289577 10768 raft_consensus.cc:2466] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 0.
I20250904 13:57:25.289398 10918 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1fd1a3b7dd454961b9ed1dbc3e0af7ef" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:25.290776 10810 leader_election.cc:304] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c; no voters: 431fe14e8b31422ba906a93b6e5f45be, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:25.295421 10810 leader_election.cc:304] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c; no voters:
W20250904 13:57:25.296707 10810 leader_election.cc:343] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.297735 10986 raft_consensus.cc:2747] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250904 13:57:25.298604 10986 raft_consensus.cc:2802] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:57:25.299043 10986 raft_consensus.cc:491] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:57:25.299476 10986 raft_consensus.cc:3058] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.325186 10971 raft_consensus.cc:695] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 LEADER]: Becoming Leader. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Running, Role: LEADER
I20250904 13:57:25.326493 10968 raft_consensus.cc:357] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.327169 10968 raft_consensus.cc:383] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.327457 10968 raft_consensus.cc:738] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 431fe14e8b31422ba906a93b6e5f45be, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.327000 10971 consensus_queue.cc:237] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.328075 10968 consensus_queue.cc:260] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.329336 10999 raft_consensus.cc:491] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:25.329857 10986 raft_consensus.cc:513] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.330667 10918 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:25.331141 10968 ts_tablet_manager.cc:1428] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be: Time spent starting tablet: real 0.061s user 0.010s sys 0.004s
I20250904 13:57:25.329926 10999 raft_consensus.cc:513] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.330668 10917 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1fd1a3b7dd454961b9ed1dbc3e0af7ef" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3"
I20250904 13:57:25.332155 10968 tablet_bootstrap.cc:492] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be: Bootstrap starting.
I20250904 13:57:25.334357 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be"
I20250904 13:57:25.335281 10768 raft_consensus.cc:3058] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.334437 10975 raft_consensus.cc:491] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20250904 13:57:25.333979 10810 leader_election.cc:343] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.337460 10975 raft_consensus.cc:513] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.340564 10975 leader_election.cc:290] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:25.338063 10986 leader_election.cc:290] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Requested vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:25.338421 10999 leader_election.cc:290] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:25.343165 10769 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6f08a2bf1d1841a19924894a1af8dcac" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:25.343165 10917 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1cd635db3064f9c94a5dbc6d620db61" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:25.343171 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1cd635db3064f9c94a5dbc6d620db61" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
W20250904 13:57:25.346328 10810 leader_election.cc:343] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.347185 10918 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3"
I20250904 13:57:25.348718 10916 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6f08a2bf1d1841a19924894a1af8dcac" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
W20250904 13:57:25.349323 10735 leader_election.cc:343] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.349329 10966 tablet_bootstrap.cc:654] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.349901 10916 raft_consensus.cc:2466] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 0.
W20250904 13:57:25.350206 10735 leader_election.cc:343] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.351720 10735 leader_election.cc:304] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
W20250904 13:57:25.352275 10810 leader_election.cc:343] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.353765 10975 raft_consensus.cc:2747] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250904 13:57:25.355263 10768 raft_consensus.cc:2466] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 1.
I20250904 13:57:25.353243 10967 tablet_bootstrap.cc:492] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: No bootstrap required, opened a new log
I20250904 13:57:25.356278 10735 leader_election.cc:304] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:25.357282 10975 raft_consensus.cc:2747] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250904 13:57:25.359246 10967 ts_tablet_manager.cc:1397] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Time spent bootstrapping tablet: real 0.119s user 0.020s sys 0.009s
W20250904 13:57:25.359421 10810 leader_election.cc:343] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.360108 10810 leader_election.cc:304] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be
I20250904 13:57:25.360917 10999 raft_consensus.cc:2802] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:57:25.361313 10999 raft_consensus.cc:491] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:57:25.361654 10999 raft_consensus.cc:3058] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.363185 10967 raft_consensus.cc:357] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.363952 10967 raft_consensus.cc:383] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.364267 10967 raft_consensus.cc:738] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.364972 10967 consensus_queue.cc:260] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.371088 10810 leader_election.cc:304] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c; no voters: 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:25.371902 10999 raft_consensus.cc:513] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.373939 10999 leader_election.cc:290] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Requested vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:25.374680 10971 raft_consensus.cc:2802] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:25.376418 10971 raft_consensus.cc:695] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 LEADER]: Becoming Leader. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Running, Role: LEADER
I20250904 13:57:25.376973 10916 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6f08a2bf1d1841a19924894a1af8dcac" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3"
I20250904 13:57:25.377641 10916 raft_consensus.cc:3058] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.377288 10971 consensus_queue.cc:237] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.382316 10664 catalog_manager.cc:5582] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130). New cstate: current_term: 1 leader_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } health_report { overall_health: UNKNOWN } } }
I20250904 13:57:25.383972 10967 ts_tablet_manager.cc:1428] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Time spent starting tablet: real 0.024s user 0.007s sys 0.001s
I20250904 13:57:25.385392 10967 tablet_bootstrap.cc:492] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Bootstrap starting.
I20250904 13:57:25.385445 10916 raft_consensus.cc:2466] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 1.
I20250904 13:57:25.389770 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6f08a2bf1d1841a19924894a1af8dcac" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be"
I20250904 13:57:25.389796 10810 leader_election.cc:304] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3; no voters:
I20250904 13:57:25.393590 10999 raft_consensus.cc:2802] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Leader election won for term 1
W20250904 13:57:25.393406 10810 leader_election.cc:343] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.394333 10999 raft_consensus.cc:695] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 LEADER]: Becoming Leader. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Running, Role: LEADER
I20250904 13:57:25.396114 10968 tablet_bootstrap.cc:654] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.397974 10999 consensus_queue.cc:237] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.403745 10966 tablet_bootstrap.cc:492] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3: No bootstrap required, opened a new log
I20250904 13:57:25.404486 10966 ts_tablet_manager.cc:1397] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3: Time spent bootstrapping tablet: real 0.134s user 0.016s sys 0.019s
I20250904 13:57:25.414939 10966 raft_consensus.cc:357] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.416059 10966 raft_consensus.cc:383] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.416538 10966 raft_consensus.cc:738] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 926518f7b9994fc288cbdff9d74323a3, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.417403 10966 consensus_queue.cc:260] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.419824 10967 tablet_bootstrap.cc:654] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.430644 10966 ts_tablet_manager.cc:1428] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3: Time spent starting tablet: real 0.024s user 0.008s sys 0.000s
I20250904 13:57:25.432277 10968 tablet_bootstrap.cc:492] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be: No bootstrap required, opened a new log
I20250904 13:57:25.433207 10967 tablet_bootstrap.cc:492] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: No bootstrap required, opened a new log
I20250904 13:57:25.434221 10967 ts_tablet_manager.cc:1397] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Time spent bootstrapping tablet: real 0.049s user 0.017s sys 0.005s
I20250904 13:57:25.435037 10966 tablet_bootstrap.cc:492] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3: Bootstrap starting.
I20250904 13:57:25.437083 10967 raft_consensus.cc:357] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.437870 10967 raft_consensus.cc:383] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.438337 10967 raft_consensus.cc:738] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.439038 10967 consensus_queue.cc:260] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.441764 10967 ts_tablet_manager.cc:1428] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250904 13:57:25.432955 10968 ts_tablet_manager.cc:1397] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be: Time spent bootstrapping tablet: real 0.101s user 0.013s sys 0.000s
I20250904 13:57:25.442898 10967 tablet_bootstrap.cc:492] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Bootstrap starting.
I20250904 13:57:25.445014 10968 raft_consensus.cc:357] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.445737 10968 raft_consensus.cc:383] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.446043 10968 raft_consensus.cc:738] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 431fe14e8b31422ba906a93b6e5f45be, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.447288 10968 consensus_queue.cc:260] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.449728 10967 tablet_bootstrap.cc:654] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.456954 10968 ts_tablet_manager.cc:1428] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20250904 13:57:25.458763 10966 tablet_bootstrap.cc:654] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.457187 10665 catalog_manager.cc:5582] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130). New cstate: current_term: 1 leader_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } health_report { overall_health: UNKNOWN } } }
I20250904 13:57:25.461550 10968 tablet_bootstrap.cc:492] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be: Bootstrap starting.
I20250904 13:57:25.460681 10665 catalog_manager.cc:5582] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130). New cstate: current_term: 1 leader_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } health_report { overall_health: UNKNOWN } } }
I20250904 13:57:25.473461 10967 tablet_bootstrap.cc:492] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: No bootstrap required, opened a new log
I20250904 13:57:25.473982 10967 ts_tablet_manager.cc:1397] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Time spent bootstrapping tablet: real 0.031s user 0.022s sys 0.004s
I20250904 13:57:25.477430 10966 tablet_bootstrap.cc:492] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3: No bootstrap required, opened a new log
I20250904 13:57:25.478078 10966 ts_tablet_manager.cc:1397] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3: Time spent bootstrapping tablet: real 0.043s user 0.008s sys 0.009s
I20250904 13:57:25.483865 10967 raft_consensus.cc:357] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.484709 10967 raft_consensus.cc:383] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.484437 10966 raft_consensus.cc:357] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.485067 10968 tablet_bootstrap.cc:654] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.485235 10966 raft_consensus.cc:383] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.485615 10966 raft_consensus.cc:738] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 926518f7b9994fc288cbdff9d74323a3, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.485030 10967 raft_consensus.cc:738] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.486521 10967 consensus_queue.cc:260] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.486464 10966 consensus_queue.cc:260] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.489104 10967 ts_tablet_manager.cc:1428] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20250904 13:57:25.490175 10967 tablet_bootstrap.cc:492] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Bootstrap starting.
I20250904 13:57:25.491576 10966 ts_tablet_manager.cc:1428] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3: Time spent starting tablet: real 0.013s user 0.009s sys 0.001s
I20250904 13:57:25.492671 10966 tablet_bootstrap.cc:492] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3: Bootstrap starting.
I20250904 13:57:25.495824 10996 raft_consensus.cc:491] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:25.497045 10967 tablet_bootstrap.cc:654] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.496536 10996 raft_consensus.cc:513] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.499253 10996 leader_election.cc:290] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:25.504913 10968 tablet_bootstrap.cc:492] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be: No bootstrap required, opened a new log
I20250904 13:57:25.504778 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1fd1a3b7dd454961b9ed1dbc3e0af7ef" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:25.505407 10968 ts_tablet_manager.cc:1397] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be: Time spent bootstrapping tablet: real 0.044s user 0.016s sys 0.008s
I20250904 13:57:25.505645 10768 raft_consensus.cc:2391] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 926518f7b9994fc288cbdff9d74323a3 in current term 1: Already voted for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in this term.
I20250904 13:57:25.506656 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1fd1a3b7dd454961b9ed1dbc3e0af7ef" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:25.508814 10968 raft_consensus.cc:357] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.509550 10968 raft_consensus.cc:383] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.509871 10968 raft_consensus.cc:738] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 431fe14e8b31422ba906a93b6e5f45be, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.510591 10968 consensus_queue.cc:260] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.513935 10884 leader_election.cc:304] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:25.515616 10996 raft_consensus.cc:3058] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.516741 10968 ts_tablet_manager.cc:1428] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20250904 13:57:25.517848 10968 tablet_bootstrap.cc:492] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be: Bootstrap starting.
I20250904 13:57:25.521287 10967 tablet_bootstrap.cc:492] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c: No bootstrap required, opened a new log
I20250904 13:57:25.521799 10967 ts_tablet_manager.cc:1397] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Time spent bootstrapping tablet: real 0.032s user 0.015s sys 0.010s
I20250904 13:57:25.524348 10967 raft_consensus.cc:357] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.525027 10967 raft_consensus.cc:383] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.525298 10967 raft_consensus.cc:738] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.525943 10967 consensus_queue.cc:260] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.526723 10966 tablet_bootstrap.cc:654] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.528379 10967 ts_tablet_manager.cc:1428] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250904 13:57:25.529328 10967 tablet_bootstrap.cc:492] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Bootstrap starting.
I20250904 13:57:25.530658 10996 raft_consensus.cc:2747] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250904 13:57:25.534435 10966 tablet_bootstrap.cc:492] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3: No bootstrap required, opened a new log
I20250904 13:57:25.534966 10966 ts_tablet_manager.cc:1397] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3: Time spent bootstrapping tablet: real 0.043s user 0.010s sys 0.002s
I20250904 13:57:25.534957 10967 tablet_bootstrap.cc:654] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.535650 10968 tablet_bootstrap.cc:654] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.538780 10966 raft_consensus.cc:357] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.539840 10966 raft_consensus.cc:383] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.540230 10966 raft_consensus.cc:738] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 926518f7b9994fc288cbdff9d74323a3, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.541323 10966 consensus_queue.cc:260] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.547416 10966 ts_tablet_manager.cc:1428] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3: Time spent starting tablet: real 0.012s user 0.008s sys 0.001s
I20250904 13:57:25.549556 10966 tablet_bootstrap.cc:492] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3: Bootstrap starting.
I20250904 13:57:25.555294 10967 tablet_bootstrap.cc:492] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: No bootstrap required, opened a new log
I20250904 13:57:25.555827 10967 ts_tablet_manager.cc:1397] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Time spent bootstrapping tablet: real 0.027s user 0.018s sys 0.005s
I20250904 13:57:25.558499 10967 raft_consensus.cc:357] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } }
I20250904 13:57:25.559428 10967 raft_consensus.cc:383] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.559504 10966 tablet_bootstrap.cc:654] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.559784 10967 raft_consensus.cc:738] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.560917 10967 consensus_queue.cc:260] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } }
I20250904 13:57:25.561751 10968 tablet_bootstrap.cc:492] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be: No bootstrap required, opened a new log
I20250904 13:57:25.562235 10968 ts_tablet_manager.cc:1397] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be: Time spent bootstrapping tablet: real 0.045s user 0.014s sys 0.001s
I20250904 13:57:25.563835 10967 ts_tablet_manager.cc:1428] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20250904 13:57:25.565039 10967 tablet_bootstrap.cc:492] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Bootstrap starting.
I20250904 13:57:25.564692 10968 raft_consensus.cc:357] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.565498 10968 raft_consensus.cc:383] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.565788 10968 raft_consensus.cc:738] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 431fe14e8b31422ba906a93b6e5f45be, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.566536 10968 consensus_queue.cc:260] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.568791 10968 ts_tablet_manager.cc:1428] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20250904 13:57:25.569737 10968 tablet_bootstrap.cc:492] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be: Bootstrap starting.
I20250904 13:57:25.571530 10967 tablet_bootstrap.cc:654] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.575706 10968 tablet_bootstrap.cc:654] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.580464 10999 raft_consensus.cc:491] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:25.581044 10999 raft_consensus.cc:513] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } }
I20250904 13:57:25.583273 10999 leader_election.cc:290] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881), 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753)
I20250904 13:57:25.584640 10996 raft_consensus.cc:491] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:25.585877 10999 raft_consensus.cc:491] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:25.585340 10916 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8718fc5fdf154903a9039ed2c48a85b7" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:25.585243 10996 raft_consensus.cc:513] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.587375 10916 raft_consensus.cc:2466] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 0.
I20250904 13:57:25.585655 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8718fc5fdf154903a9039ed2c48a85b7" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:25.587117 10999 raft_consensus.cc:513] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.588938 10996 leader_election.cc:290] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
W20250904 13:57:25.589303 10810 leader_election.cc:343] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.590147 10999 leader_election.cc:290] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:25.590144 10810 leader_election.cc:304] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be
I20250904 13:57:25.591104 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "27b15f5a5a6f481cbb1af6df1053357a" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:25.591838 10844 raft_consensus.cc:2466] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 926518f7b9994fc288cbdff9d74323a3 in term 0.
I20250904 13:57:25.592301 10999 raft_consensus.cc:2802] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:57:25.592731 10999 raft_consensus.cc:491] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:57:25.593114 10999 raft_consensus.cc:3058] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.594415 10769 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "27b15f5a5a6f481cbb1af6df1053357a" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:25.594703 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5c80f91315f84cf49f62445010a99d83" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:25.595126 10769 raft_consensus.cc:2466] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 926518f7b9994fc288cbdff9d74323a3 in term 0.
I20250904 13:57:25.595388 10768 raft_consensus.cc:2466] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 0.
I20250904 13:57:25.596520 10884 leader_election.cc:304] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be, 926518f7b9994fc288cbdff9d74323a3; no voters:
I20250904 13:57:25.596932 10968 tablet_bootstrap.cc:492] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be: No bootstrap required, opened a new log
I20250904 13:57:25.597427 10968 ts_tablet_manager.cc:1397] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be: Time spent bootstrapping tablet: real 0.028s user 0.015s sys 0.000s
I20250904 13:57:25.596771 10810 leader_election.cc:304] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c; no voters:
I20250904 13:57:25.598335 10971 raft_consensus.cc:2802] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:57:25.598721 10971 raft_consensus.cc:491] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:57:25.599051 10971 raft_consensus.cc:3058] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.598878 10916 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5c80f91315f84cf49f62445010a99d83" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:25.599682 10916 raft_consensus.cc:2466] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 0.
I20250904 13:57:25.600574 10968 raft_consensus.cc:357] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } }
I20250904 13:57:25.600951 10996 raft_consensus.cc:2802] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:57:25.601235 10968 raft_consensus.cc:383] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.601393 10996 raft_consensus.cc:491] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:57:25.601584 10968 raft_consensus.cc:738] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 431fe14e8b31422ba906a93b6e5f45be, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.601677 10967 tablet_bootstrap.cc:492] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: No bootstrap required, opened a new log
I20250904 13:57:25.601838 10996 raft_consensus.cc:3058] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.602243 10967 ts_tablet_manager.cc:1397] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Time spent bootstrapping tablet: real 0.037s user 0.013s sys 0.000s
I20250904 13:57:25.602376 10968 consensus_queue.cc:260] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } }
I20250904 13:57:25.604628 10968 ts_tablet_manager.cc:1428] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20250904 13:57:25.604867 10967 raft_consensus.cc:357] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.605561 10967 raft_consensus.cc:383] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.605840 10967 raft_consensus.cc:738] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.606154 10968 tablet_bootstrap.cc:492] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be: Bootstrap starting.
I20250904 13:57:25.606518 10967 consensus_queue.cc:260] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.608305 10971 raft_consensus.cc:513] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.614033 10971 leader_election.cc:290] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Requested vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:25.614466 10968 tablet_bootstrap.cc:654] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.609040 10999 raft_consensus.cc:513] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } }
I20250904 13:57:25.616672 10999 leader_election.cc:290] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Requested vote from peers 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881), 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753)
I20250904 13:57:25.619680 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5c80f91315f84cf49f62445010a99d83" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be"
I20250904 13:57:25.622752 10768 raft_consensus.cc:3058] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.623100 10918 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8718fc5fdf154903a9039ed2c48a85b7" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3"
I20250904 13:57:25.623750 10918 raft_consensus.cc:3058] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.624837 10967 ts_tablet_manager.cc:1428] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20250904 13:57:25.621483 10916 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5c80f91315f84cf49f62445010a99d83" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3"
I20250904 13:57:25.626746 10916 raft_consensus.cc:3058] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.621948 10769 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8718fc5fdf154903a9039ed2c48a85b7" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be"
I20250904 13:57:25.630405 10769 raft_consensus.cc:3058] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.626798 10971 raft_consensus.cc:491] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:25.634534 10971 raft_consensus.cc:513] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.636552 10968 tablet_bootstrap.cc:492] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be: No bootstrap required, opened a new log
I20250904 13:57:25.636632 10971 leader_election.cc:290] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:25.637125 10968 ts_tablet_manager.cc:1397] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be: Time spent bootstrapping tablet: real 0.031s user 0.009s sys 0.019s
I20250904 13:57:25.637974 10767 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1cd635db3064f9c94a5dbc6d620db61" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:25.638407 10917 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1cd635db3064f9c94a5dbc6d620db61" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:25.638708 10767 raft_consensus.cc:2466] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 0.
I20250904 13:57:25.640108 10810 leader_election.cc:304] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c; no voters:
W20250904 13:57:25.641281 10810 leader_election.cc:343] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.641606 10999 raft_consensus.cc:2802] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:57:25.642001 10999 raft_consensus.cc:491] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:57:25.642385 10999 raft_consensus.cc:3058] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.642493 10968 raft_consensus.cc:357] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.643342 10968 raft_consensus.cc:383] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.646067 10986 consensus_queue.cc:1035] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Connected to new peer: Peer: permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250904 13:57:25.646983 10918 raft_consensus.cc:2466] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 1.
W20250904 13:57:25.647666 10810 consensus_peers.cc:489] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c -> Peer 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881): Couldn't send request to peer 926518f7b9994fc288cbdff9d74323a3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20250904 13:57:25.641855 10769 raft_consensus.cc:2466] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 1.
I20250904 13:57:25.648912 10810 leader_election.cc:304] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3; no voters:
I20250904 13:57:25.645947 10916 raft_consensus.cc:2466] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 1.
I20250904 13:57:25.650066 10971 raft_consensus.cc:2802] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:25.643666 10968 raft_consensus.cc:738] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 431fe14e8b31422ba906a93b6e5f45be, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.650892 10768 raft_consensus.cc:2466] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 1.
I20250904 13:57:25.651574 10968 consensus_queue.cc:260] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.649351 10986 raft_consensus.cc:491] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:25.654279 10968 ts_tablet_manager.cc:1428] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be: Time spent starting tablet: real 0.017s user 0.008s sys 0.001s
I20250904 13:57:25.653954 10986 raft_consensus.cc:513] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.654956 10810 leader_election.cc:304] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c; no voters:
I20250904 13:57:25.656172 10986 leader_election.cc:290] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:25.657538 10986 raft_consensus.cc:2802] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:25.658615 10986 raft_consensus.cc:695] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 LEADER]: Becoming Leader. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Running, Role: LEADER
I20250904 13:57:25.660295 10999 raft_consensus.cc:513] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.660195 10996 raft_consensus.cc:513] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.662225 10999 leader_election.cc:290] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Requested vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:25.663245 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "27b15f5a5a6f481cbb1af6df1053357a" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be"
I20250904 13:57:25.664112 10966 tablet_bootstrap.cc:492] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3: No bootstrap required, opened a new log
I20250904 13:57:25.664909 10966 ts_tablet_manager.cc:1397] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3: Time spent bootstrapping tablet: real 0.116s user 0.010s sys 0.009s
I20250904 13:57:25.665027 10986 consensus_queue.cc:237] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.667662 10966 raft_consensus.cc:357] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.668533 10966 raft_consensus.cc:383] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.668865 10966 raft_consensus.cc:738] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 926518f7b9994fc288cbdff9d74323a3, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.664177 10768 raft_consensus.cc:3058] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.670511 10999 raft_consensus.cc:491] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:25.671229 10999 raft_consensus.cc:513] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.669677 10966 consensus_queue.cc:260] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.673215 10999 leader_election.cc:290] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:25.675410 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "27b15f5a5a6f481cbb1af6df1053357a" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c"
I20250904 13:57:25.678727 10844 raft_consensus.cc:3058] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.675040 10996 leader_election.cc:290] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 election: Requested vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:25.681267 10966 ts_tablet_manager.cc:1428] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3: Time spent starting tablet: real 0.016s user 0.007s sys 0.000s
I20250904 13:57:25.675423 10971 raft_consensus.cc:695] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 LEADER]: Becoming Leader. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Running, Role: LEADER
I20250904 13:57:25.682586 10966 tablet_bootstrap.cc:492] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3: Bootstrap starting.
I20250904 13:57:25.684156 10971 consensus_queue.cc:237] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } }
I20250904 13:57:25.688156 10844 raft_consensus.cc:2466] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 926518f7b9994fc288cbdff9d74323a3 in term 1.
I20250904 13:57:25.689260 10966 tablet_bootstrap.cc:654] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.691136 10884 leader_election.cc:304] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3; no voters:
I20250904 13:57:25.693843 10996 raft_consensus.cc:2802] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:25.694830 10996 raft_consensus.cc:695] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [term 1 LEADER]: Becoming Leader. State: Replica: 926518f7b9994fc288cbdff9d74323a3, State: Running, Role: LEADER
I20250904 13:57:25.695845 10996 consensus_queue.cc:237] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.700487 10999 raft_consensus.cc:491] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:25.701048 10999 raft_consensus.cc:513] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.703120 10999 leader_election.cc:290] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:25.707237 10769 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1d2da9bd6e0405fa033babd924eb743" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:25.708210 10769 raft_consensus.cc:2466] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 0.
I20250904 13:57:25.713588 10810 leader_election.cc:304] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c; no voters:
I20250904 13:57:25.716580 10765 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7cc86826b94e49289260ba78159669db" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:25.716056 10768 raft_consensus.cc:2466] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 926518f7b9994fc288cbdff9d74323a3 in term 1.
I20250904 13:57:25.714329 10767 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8f248691b38b4c76add5969ac40fbdc0" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:25.717439 10765 raft_consensus.cc:2466] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 0.
I20250904 13:57:25.717687 10767 raft_consensus.cc:2466] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 0.
I20250904 13:57:25.718806 10999 raft_consensus.cc:2802] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:57:25.719255 10999 raft_consensus.cc:491] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:57:25.719197 10769 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1cd635db3064f9c94a5dbc6d620db61" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be"
I20250904 13:57:25.719954 10769 raft_consensus.cc:3058] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.720886 10966 tablet_bootstrap.cc:492] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3: No bootstrap required, opened a new log
I20250904 13:57:25.721517 10918 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1d2da9bd6e0405fa033babd924eb743" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:25.729403 10966 ts_tablet_manager.cc:1397] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3: Time spent bootstrapping tablet: real 0.047s user 0.012s sys 0.001s
I20250904 13:57:25.729449 10999 raft_consensus.cc:3058] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.722163 10917 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8f248691b38b4c76add5969ac40fbdc0" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:25.726574 10915 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7cc86826b94e49289260ba78159669db" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:25.729523 10918 raft_consensus.cc:2466] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 0.
I20250904 13:57:25.721814 10916 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1cd635db3064f9c94a5dbc6d620db61" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3"
I20250904 13:57:25.728920 10769 raft_consensus.cc:2466] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 1.
I20250904 13:57:25.733350 10915 raft_consensus.cc:2466] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 0.
I20250904 13:57:25.737139 10966 raft_consensus.cc:357] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.738343 10966 raft_consensus.cc:383] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.738790 10966 raft_consensus.cc:738] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 926518f7b9994fc288cbdff9d74323a3, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.739625 10966 consensus_queue.cc:260] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.743090 10768 raft_consensus.cc:3058] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.743844 10665 catalog_manager.cc:5582] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130). New cstate: current_term: 1 leader_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } health_report { overall_health: UNKNOWN } } }
I20250904 13:57:25.741973 10966 ts_tablet_manager.cc:1428] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20250904 13:57:25.745602 10810 leader_election.cc:304] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c; no voters:
I20250904 13:57:25.748701 10966 tablet_bootstrap.cc:492] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3: Bootstrap starting.
I20250904 13:57:25.753283 10986 raft_consensus.cc:2802] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:57:25.753811 10986 raft_consensus.cc:491] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:57:25.754506 10986 raft_consensus.cc:3058] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.754472 10810 leader_election.cc:304] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c; no voters:
I20250904 13:57:25.763821 11022 raft_consensus.cc:2802] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:57:25.764448 11022 raft_consensus.cc:491] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:57:25.764938 11022 raft_consensus.cc:3058] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.763337 10810 leader_election.cc:304] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c; no voters:
I20250904 13:57:25.770149 10999 raft_consensus.cc:513] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.756552 10971 consensus_queue.cc:1035] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Connected to new peer: Peer: permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20250904 13:57:25.772945 11021 raft_consensus.cc:2802] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:25.778698 10666 catalog_manager.cc:5582] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131). New cstate: current_term: 1 leader_uuid: "926518f7b9994fc288cbdff9d74323a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } health_report { overall_health: HEALTHY } } }
I20250904 13:57:25.781168 11022 raft_consensus.cc:513] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.783792 11022 leader_election.cc:290] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Requested vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:25.784353 11022 consensus_queue.cc:1035] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Connected to new peer: Peer: permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:57:25.793138 10966 tablet_bootstrap.cc:654] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:25.796051 11021 raft_consensus.cc:695] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 LEADER]: Becoming Leader. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Running, Role: LEADER
I20250904 13:57:25.792938 10986 raft_consensus.cc:513] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.795033 10999 leader_election.cc:290] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Requested vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:25.802063 11021 consensus_queue.cc:237] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.802441 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8f248691b38b4c76add5969ac40fbdc0" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be"
I20250904 13:57:25.803274 10768 raft_consensus.cc:3058] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.805775 10769 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1d2da9bd6e0405fa033babd924eb743" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be"
I20250904 13:57:25.806452 10769 raft_consensus.cc:3058] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.806788 10916 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8f248691b38b4c76add5969ac40fbdc0" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3"
I20250904 13:57:25.812569 10768 raft_consensus.cc:2466] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 1.
I20250904 13:57:25.814518 10986 leader_election.cc:290] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Requested vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
W20250904 13:57:25.814627 10810 leader_election.cc:343] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.819074 10916 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1d2da9bd6e0405fa033babd924eb743" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3"
I20250904 13:57:25.819729 10916 raft_consensus.cc:3058] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Advancing to term 1
W20250904 13:57:25.819603 10810 leader_election.cc:343] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.823287 11030 consensus_queue.cc:1035] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Connected to new peer: Peer: permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20250904 13:57:25.825860 10810 leader_election.cc:343] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:25.816076 10769 raft_consensus.cc:2466] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 1.
I20250904 13:57:25.828101 10916 raft_consensus.cc:2466] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 1.
I20250904 13:57:25.829598 11030 consensus_queue.cc:1035] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Connected to new peer: Peer: permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250904 13:57:25.831758 10810 leader_election.cc:304] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c; no voters: 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:25.832726 11030 raft_consensus.cc:2802] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:25.833213 11030 raft_consensus.cc:695] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 LEADER]: Becoming Leader. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Running, Role: LEADER
I20250904 13:57:25.834378 11030 consensus_queue.cc:237] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.839339 10810 leader_election.cc:304] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c; no voters:
I20250904 13:57:25.843258 11021 raft_consensus.cc:2802] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:25.844872 11021 raft_consensus.cc:695] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 LEADER]: Becoming Leader. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Running, Role: LEADER
I20250904 13:57:25.845805 11021 consensus_queue.cc:237] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.850600 10966 tablet_bootstrap.cc:492] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3: No bootstrap required, opened a new log
I20250904 13:57:25.851034 10966 ts_tablet_manager.cc:1397] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3: Time spent bootstrapping tablet: real 0.103s user 0.014s sys 0.025s
I20250904 13:57:25.853307 10766 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7cc86826b94e49289260ba78159669db" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be"
I20250904 13:57:25.854045 10766 raft_consensus.cc:3058] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.856660 10916 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7cc86826b94e49289260ba78159669db" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3"
I20250904 13:57:25.857348 10916 raft_consensus.cc:3058] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.869325 10966 raft_consensus.cc:357] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.870517 10966 raft_consensus.cc:383] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:25.870886 10966 raft_consensus.cc:738] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 926518f7b9994fc288cbdff9d74323a3, State: Initialized, Role: FOLLOWER
I20250904 13:57:25.877090 10966 consensus_queue.cc:260] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.887265 10996 raft_consensus.cc:491] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:25.887838 10996 raft_consensus.cc:513] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.888015 10666 catalog_manager.cc:5582] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130). New cstate: current_term: 1 leader_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } health_report { overall_health: UNKNOWN } } }
I20250904 13:57:25.890051 10916 raft_consensus.cc:2466] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 1.
I20250904 13:57:25.890468 10996 leader_election.cc:290] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:25.890328 10666 catalog_manager.cc:5582] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130). New cstate: current_term: 1 leader_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } health_report { overall_health: UNKNOWN } } }
I20250904 13:57:25.893175 10765 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:25.895032 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:25.896813 10966 ts_tablet_manager.cc:1428] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3: Time spent starting tablet: real 0.046s user 0.008s sys 0.002s
I20250904 13:57:25.900357 10884 leader_election.cc:304] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:25.901335 10996 raft_consensus.cc:2747] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250904 13:57:25.897667 10810 leader_election.cc:304] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3; no voters:
I20250904 13:57:25.901729 10973 raft_consensus.cc:491] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:25.902475 10986 raft_consensus.cc:2802] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:25.902357 10973 raft_consensus.cc:513] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.903026 10986 raft_consensus.cc:695] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 LEADER]: Becoming Leader. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Running, Role: LEADER
I20250904 13:57:25.905467 10986 consensus_queue.cc:237] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:25.911465 10973 leader_election.cc:290] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:25.899349 10766 raft_consensus.cc:2466] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in term 1.
I20250904 13:57:25.923230 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1cd635db3064f9c94a5dbc6d620db61" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:25.924794 11022 consensus_queue.cc:1035] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Connected to new peer: Peer: permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20250904 13:57:25.929629 10916 raft_consensus.cc:3058] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.950038 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1cd635db3064f9c94a5dbc6d620db61" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:25.960556 10768 raft_consensus.cc:2391] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 926518f7b9994fc288cbdff9d74323a3 in current term 1: Already voted for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in this term.
I20250904 13:57:25.963410 10884 leader_election.cc:304] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:25.964344 10973 raft_consensus.cc:3058] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:25.974958 11022 consensus_queue.cc:1035] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Connected to new peer: Peer: permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250904 13:57:25.984160 10973 raft_consensus.cc:2747] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250904 13:57:25.993119 10973 consensus_queue.cc:1035] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:57:25.994556 11030 consensus_queue.cc:1035] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Connected to new peer: Peer: permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:57:26.024791 10664 catalog_manager.cc:5582] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130). New cstate: current_term: 1 leader_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } health_report { overall_health: UNKNOWN } } }
I20250904 13:57:26.026103 11030 consensus_queue.cc:1035] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Connected to new peer: Peer: permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:57:26.027019 10664 catalog_manager.cc:5582] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130). New cstate: current_term: 1 leader_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } health_report { overall_health: UNKNOWN } } }
I20250904 13:57:26.039861 10664 catalog_manager.cc:5582] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130). New cstate: current_term: 1 leader_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } health_report { overall_health: UNKNOWN } } }
I20250904 13:57:26.070662 10996 consensus_queue.cc:1035] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:57:26.098229 10973 raft_consensus.cc:491] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:26.098776 10973 raft_consensus.cc:513] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:26.100730 10973 leader_election.cc:290] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:26.101253 11030 consensus_queue.cc:1035] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Connected to new peer: Peer: permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:57:26.104732 10765 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8f248691b38b4c76add5969ac40fbdc0" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:26.105594 10765 raft_consensus.cc:2391] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 926518f7b9994fc288cbdff9d74323a3 in current term 1: Already voted for candidate 6ddeb67cfc6e4f5ba3028e51fcdd545c in this term.
I20250904 13:57:26.108130 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8f248691b38b4c76add5969ac40fbdc0" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:26.109843 10986 consensus_queue.cc:1035] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Connected to new peer: Peer: permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:57:26.111230 10884 leader_election.cc:304] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:26.112453 10973 raft_consensus.cc:3058] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:26.126089 10973 raft_consensus.cc:2747] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250904 13:57:26.141961 10971 consensus_queue.cc:1035] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Connected to new peer: Peer: permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250904 13:57:26.150804 10971 consensus_queue.cc:1035] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Connected to new peer: Peer: permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250904 13:57:26.181320 10971 consensus_queue.cc:1035] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Connected to new peer: Peer: permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:57:26.204682 10971 consensus_queue.cc:1035] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Connected to new peer: Peer: permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250904 13:57:26.218573 11030 consensus_queue.cc:1035] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Connected to new peer: Peer: permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:57:26.236996 10986 consensus_queue.cc:1035] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Connected to new peer: Peer: permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250904 13:57:26.265050 10634 tablet_server_quiescing-itest.cc:159] Tablet 8718fc5fdf154903a9039ed2c48a85b7 has leader 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:26.270731 10634 tablet_server_quiescing-itest.cc:159] Tablet 7cc86826b94e49289260ba78159669db has leader 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:26.276273 10634 tablet_server_quiescing-itest.cc:159] Tablet 5c80f91315f84cf49f62445010a99d83 has leader 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:26.281486 10634 tablet_server_quiescing-itest.cc:159] Tablet a1d2da9bd6e0405fa033babd924eb743 has leader 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:26.287367 10634 tablet_server_quiescing-itest.cc:159] Tablet a1cd635db3064f9c94a5dbc6d620db61 has leader 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:26.292618 10634 tablet_server_quiescing-itest.cc:159] Tablet 6f08a2bf1d1841a19924894a1af8dcac has leader 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:26.297782 10634 tablet_server_quiescing-itest.cc:159] Tablet 1fd1a3b7dd454961b9ed1dbc3e0af7ef has leader 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:26.303001 10634 tablet_server_quiescing-itest.cc:159] Tablet bb8182b451f946b78811668ce1d0ce37 has leader 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:26.321411 10634 tablet_server_quiescing-itest.cc:159] Tablet 27b15f5a5a6f481cbb1af6df1053357a has leader 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:26.326798 10634 tablet_server_quiescing-itest.cc:159] Tablet 8f248691b38b4c76add5969ac40fbdc0 has leader 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:26.327113 10634 tablet_server_quiescing-itest.cc:163] Quiescing ts 431fe14e8b31422ba906a93b6e5f45be
W20250904 13:57:26.363274 10973 fault_injection.cc:43] FAULT INJECTION ENABLED!
W20250904 13:57:26.363601 10973 fault_injection.cc:44] THIS SERVER MAY CRASH!
I20250904 13:57:26.712301 10975 raft_consensus.cc:491] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:26.712901 10975 raft_consensus.cc:513] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:26.714898 10975 leader_election.cc:290] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:26.716166 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1fd1a3b7dd454961b9ed1dbc3e0af7ef" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:26.716166 10915 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1fd1a3b7dd454961b9ed1dbc3e0af7ef" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:26.717904 10996 raft_consensus.cc:491] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:26.717967 10735 leader_election.cc:304] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:26.718518 10996 raft_consensus.cc:513] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:26.720551 10996 leader_election.cc:290] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:26.721426 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:26.722934 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:26.724651 10884 leader_election.cc:304] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:26.724865 10975 raft_consensus.cc:2747] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:26.725363 10996 raft_consensus.cc:2747] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:26.753895 10975 raft_consensus.cc:491] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:26.754528 10975 raft_consensus.cc:513] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:26.756302 10975 leader_election.cc:290] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:26.757529 10915 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8f248691b38b4c76add5969ac40fbdc0" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:26.757476 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8f248691b38b4c76add5969ac40fbdc0" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:26.759336 10735 leader_election.cc:304] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:26.760361 10975 raft_consensus.cc:2747] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:26.801018 10996 raft_consensus.cc:491] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:26.801568 10996 raft_consensus.cc:513] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:26.803354 10996 leader_election.cc:290] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:26.805220 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1cd635db3064f9c94a5dbc6d620db61" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:26.805493 10767 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1cd635db3064f9c94a5dbc6d620db61" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:26.807107 10884 leader_election.cc:304] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:26.807794 10996 raft_consensus.cc:2747] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:26.903332 10996 raft_consensus.cc:491] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:26.904006 10996 raft_consensus.cc:513] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:26.906000 10996 leader_election.cc:290] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:26.907629 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7cc86826b94e49289260ba78159669db" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:26.907819 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7cc86826b94e49289260ba78159669db" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:26.909504 10884 leader_election.cc:304] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:26.910221 10996 raft_consensus.cc:2747] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:26.936645 10975 raft_consensus.cc:491] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:26.937357 10975 raft_consensus.cc:513] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:26.939328 10975 leader_election.cc:290] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:26.940560 10915 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:26.940524 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:26.942251 10735 leader_election.cc:304] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:26.943096 10975 raft_consensus.cc:2747] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:26.949707 10996 raft_consensus.cc:491] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:26.950245 10996 raft_consensus.cc:513] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:26.952009 10996 leader_election.cc:290] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:26.952176 10999 raft_consensus.cc:491] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 926518f7b9994fc288cbdff9d74323a3)
I20250904 13:57:26.952713 10999 raft_consensus.cc:513] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:26.952951 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5c80f91315f84cf49f62445010a99d83" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:26.953696 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5c80f91315f84cf49f62445010a99d83" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:26.955073 10999 leader_election.cc:290] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:26.955859 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "27b15f5a5a6f481cbb1af6df1053357a" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:26.955020 10884 leader_election.cc:304] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:26.956662 10996 raft_consensus.cc:2747] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:26.957652 10915 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "27b15f5a5a6f481cbb1af6df1053357a" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:26.959201 10810 leader_election.cc:304] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c; no voters: 431fe14e8b31422ba906a93b6e5f45be, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:26.959975 10999 raft_consensus.cc:2747] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.000394 10996 raft_consensus.cc:491] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.000916 10996 raft_consensus.cc:513] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } }
I20250904 13:57:27.002527 10996 leader_election.cc:290] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753)
I20250904 13:57:27.003499 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8718fc5fdf154903a9039ed2c48a85b7" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.003690 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8718fc5fdf154903a9039ed2c48a85b7" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:27.004951 10884 leader_election.cc:304] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:27.005651 10996 raft_consensus.cc:2747] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.044377 10975 raft_consensus.cc:491] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.044925 10975 raft_consensus.cc:513] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.046509 10975 leader_election.cc:290] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:27.047521 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7cc86826b94e49289260ba78159669db" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.047703 10915 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7cc86826b94e49289260ba78159669db" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:27.049010 10735 leader_election.cc:304] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:27.049666 10975 raft_consensus.cc:2747] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.067836 10975 raft_consensus.cc:491] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.068413 10975 raft_consensus.cc:513] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.070003 10975 leader_election.cc:290] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:27.070894 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1d2da9bd6e0405fa033babd924eb743" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.071059 10915 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1d2da9bd6e0405fa033babd924eb743" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:27.072886 10735 leader_election.cc:304] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:27.073657 10975 raft_consensus.cc:2747] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.195194 10975 raft_consensus.cc:491] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.195690 10975 raft_consensus.cc:513] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.197285 10975 leader_election.cc:290] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:27.198218 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1cd635db3064f9c94a5dbc6d620db61" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.198371 10915 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1cd635db3064f9c94a5dbc6d620db61" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:27.199790 10735 leader_election.cc:304] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:27.200456 10975 raft_consensus.cc:2747] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.204228 10975 raft_consensus.cc:491] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 926518f7b9994fc288cbdff9d74323a3)
I20250904 13:57:27.204777 10975 raft_consensus.cc:513] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.206329 10975 leader_election.cc:290] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:27.207186 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "27b15f5a5a6f481cbb1af6df1053357a" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.207263 10915 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "27b15f5a5a6f481cbb1af6df1053357a" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:27.208664 10735 leader_election.cc:304] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:27.209331 10975 raft_consensus.cc:2747] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.209844 10996 raft_consensus.cc:491] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.210245 10996 raft_consensus.cc:513] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.211762 10996 leader_election.cc:290] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:27.212620 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1fd1a3b7dd454961b9ed1dbc3e0af7ef" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:27.212838 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1fd1a3b7dd454961b9ed1dbc3e0af7ef" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.214465 10884 leader_election.cc:304] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:27.215127 10996 raft_consensus.cc:2747] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.215651 10973 raft_consensus.cc:491] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.216070 10973 raft_consensus.cc:513] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.217514 10973 leader_election.cc:290] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:27.218351 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8f248691b38b4c76add5969ac40fbdc0" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:27.218480 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8f248691b38b4c76add5969ac40fbdc0" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.219750 10884 leader_election.cc:304] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:27.220374 10973 raft_consensus.cc:2747] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.260589 10975 raft_consensus.cc:491] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.261119 10975 raft_consensus.cc:513] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.262984 10975 leader_election.cc:290] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:27.263883 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5c80f91315f84cf49f62445010a99d83" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.263936 10915 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5c80f91315f84cf49f62445010a99d83" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:27.265650 10735 leader_election.cc:304] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:27.266556 10975 raft_consensus.cc:2747] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.318359 10973 raft_consensus.cc:491] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.318866 10973 raft_consensus.cc:513] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.321105 10973 leader_election.cc:290] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:27.321980 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:27.322175 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.323930 10884 leader_election.cc:304] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:27.324581 10973 raft_consensus.cc:2747] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.378593 10996 raft_consensus.cc:491] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.379063 10996 raft_consensus.cc:513] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.380775 10996 leader_election.cc:290] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:27.381585 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1cd635db3064f9c94a5dbc6d620db61" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:27.381819 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1cd635db3064f9c94a5dbc6d620db61" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.383255 10884 leader_election.cc:304] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:27.383909 10996 raft_consensus.cc:2747] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.384513 10975 raft_consensus.cc:491] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.384918 10975 raft_consensus.cc:513] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.386437 10975 leader_election.cc:290] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:27.387279 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1fd1a3b7dd454961b9ed1dbc3e0af7ef" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.387411 10915 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1fd1a3b7dd454961b9ed1dbc3e0af7ef" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:27.388686 10735 leader_election.cc:304] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:27.389331 10975 raft_consensus.cc:2747] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.469854 10996 raft_consensus.cc:491] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.470403 10996 raft_consensus.cc:513] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.471909 10996 leader_election.cc:290] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:27.472743 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1d2da9bd6e0405fa033babd924eb743" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:27.472955 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1d2da9bd6e0405fa033babd924eb743" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.474184 10884 leader_election.cc:304] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:27.474835 10996 raft_consensus.cc:2747] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.543632 10996 raft_consensus.cc:491] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.544134 10996 raft_consensus.cc:513] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.545763 10996 leader_election.cc:290] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:27.546933 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5c80f91315f84cf49f62445010a99d83" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:27.546967 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5c80f91315f84cf49f62445010a99d83" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.548512 10884 leader_election.cc:304] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:27.549137 10996 raft_consensus.cc:2747] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.551736 10975 raft_consensus.cc:491] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.552161 10975 raft_consensus.cc:513] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.553661 10975 leader_election.cc:290] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:27.554652 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.554756 10915 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:27.556360 10735 leader_election.cc:304] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:27.556982 10975 raft_consensus.cc:2747] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.570262 10996 raft_consensus.cc:491] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.570895 10996 raft_consensus.cc:513] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.572515 10996 leader_election.cc:290] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:27.573437 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7cc86826b94e49289260ba78159669db" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:27.573621 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7cc86826b94e49289260ba78159669db" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.574916 10884 leader_election.cc:304] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:27.575599 10996 raft_consensus.cc:2747] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.624212 10999 raft_consensus.cc:491] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 926518f7b9994fc288cbdff9d74323a3)
I20250904 13:57:27.624747 10999 raft_consensus.cc:513] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.626484 10999 leader_election.cc:290] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:27.627207 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "27b15f5a5a6f481cbb1af6df1053357a" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:27.627456 10915 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "27b15f5a5a6f481cbb1af6df1053357a" candidate_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:27.628608 10810 leader_election.cc:304] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c; no voters: 431fe14e8b31422ba906a93b6e5f45be, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:27.629279 10999 raft_consensus.cc:2747] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.633742 10996 raft_consensus.cc:491] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.634297 10996 raft_consensus.cc:513] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.635890 10996 leader_election.cc:290] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:27.636854 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6f08a2bf1d1841a19924894a1af8dcac" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:27.636940 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6f08a2bf1d1841a19924894a1af8dcac" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.638386 10884 leader_election.cc:304] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:27.639074 10996 raft_consensus.cc:2747] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.656948 10996 raft_consensus.cc:491] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.657488 10996 raft_consensus.cc:513] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.659276 10996 leader_election.cc:290] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:27.660250 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:27.660406 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.661793 10884 leader_election.cc:304] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:27.662525 10996 raft_consensus.cc:2747] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.680881 10975 raft_consensus.cc:491] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.681417 10975 raft_consensus.cc:513] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.683019 10975 leader_election.cc:290] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:27.684098 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8f248691b38b4c76add5969ac40fbdc0" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.684160 10915 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8f248691b38b4c76add5969ac40fbdc0" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:27.685774 10735 leader_election.cc:304] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:27.686431 10975 raft_consensus.cc:2747] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.707862 10975 raft_consensus.cc:491] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.708303 10975 raft_consensus.cc:513] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.709888 10975 leader_election.cc:290] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:27.710723 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7cc86826b94e49289260ba78159669db" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.711024 10915 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7cc86826b94e49289260ba78159669db" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:27.712224 10735 leader_election.cc:304] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:27.712854 10975 raft_consensus.cc:2747] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.715054 10975 raft_consensus.cc:491] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.715457 10975 raft_consensus.cc:513] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.717013 10975 leader_election.cc:290] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:27.717716 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1d2da9bd6e0405fa033babd924eb743" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.717947 10915 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1d2da9bd6e0405fa033babd924eb743" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:27.719161 10735 leader_election.cc:304] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:27.719820 10975 raft_consensus.cc:2747] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.748414 10975 raft_consensus.cc:491] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.748883 10975 raft_consensus.cc:513] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.750425 10975 leader_election.cc:290] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:27.751286 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6f08a2bf1d1841a19924894a1af8dcac" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.751446 10915 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6f08a2bf1d1841a19924894a1af8dcac" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:27.752653 10735 leader_election.cc:304] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:27.753297 10975 raft_consensus.cc:2747] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.879104 10996 raft_consensus.cc:491] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.879617 10996 raft_consensus.cc:513] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } }
I20250904 13:57:27.881237 10996 leader_election.cc:290] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753)
I20250904 13:57:27.882332 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8718fc5fdf154903a9039ed2c48a85b7" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.882441 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8718fc5fdf154903a9039ed2c48a85b7" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:27.883872 10884 leader_election.cc:304] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:27.884560 10996 raft_consensus.cc:2747] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.903249 10975 raft_consensus.cc:491] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.903761 10975 raft_consensus.cc:513] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } }
I20250904 13:57:27.905386 10975 leader_election.cc:290] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:27.906445 10915 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8718fc5fdf154903a9039ed2c48a85b7" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:27.906582 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8718fc5fdf154903a9039ed2c48a85b7" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.907042 10915 raft_consensus.cc:2466] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 431fe14e8b31422ba906a93b6e5f45be in term 1.
I20250904 13:57:27.907987 10735 leader_election.cc:304] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be, 926518f7b9994fc288cbdff9d74323a3; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:27.908641 10975 raft_consensus.cc:2802] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250904 13:57:27.908913 10975 raft_consensus.cc:491] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.909186 10975 raft_consensus.cc:3058] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Advancing to term 2
I20250904 13:57:27.913990 10975 raft_consensus.cc:513] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } }
I20250904 13:57:27.915711 10975 leader_election.cc:290] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 election: Requested vote from peers 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:27.916394 10918 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8718fc5fdf154903a9039ed2c48a85b7" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3"
I20250904 13:57:27.916704 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8718fc5fdf154903a9039ed2c48a85b7" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c"
I20250904 13:57:27.917038 10918 raft_consensus.cc:3058] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Advancing to term 2
I20250904 13:57:27.922024 10918 raft_consensus.cc:2466] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 431fe14e8b31422ba906a93b6e5f45be in term 2.
I20250904 13:57:27.923053 10735 leader_election.cc:304] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be, 926518f7b9994fc288cbdff9d74323a3; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:27.923614 10975 raft_consensus.cc:491] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:27.924140 10975 raft_consensus.cc:513] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:27.924612 11066 raft_consensus.cc:2802] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [term 2 FOLLOWER]: Leader election won for term 2
I20250904 13:57:27.925803 11066 raft_consensus.cc:695] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [term 2 LEADER]: Becoming Leader. State: Replica: 431fe14e8b31422ba906a93b6e5f45be, State: Running, Role: LEADER
I20250904 13:57:27.926731 11066 consensus_queue.cc:237] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } }
I20250904 13:57:27.927181 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:27.928423 10975 leader_election.cc:290] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:27.928530 10918 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:27.929723 10735 leader_election.cc:304] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:27.930663 11066 raft_consensus.cc:2747] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:27.934635 10664 catalog_manager.cc:5582] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be reported cstate change: term changed from 1 to 2, leader changed from 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130) to 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129). New cstate: current_term: 2 leader_uuid: "431fe14e8b31422ba906a93b6e5f45be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } health_report { overall_health: HEALTHY } } }
I20250904 13:57:27.940025 10768 raft_consensus.cc:1238] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [term 2 LEADER]: Rejecting Update request from peer 6ddeb67cfc6e4f5ba3028e51fcdd545c for earlier term 1. Current term is 2. Ops: []
I20250904 13:57:27.941619 11021 consensus_queue.cc:1046] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250904 13:57:27.942464 11021 consensus_queue.cc:1230] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Peer 431fe14e8b31422ba906a93b6e5f45be log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20250904 13:57:27.946071 10918 raft_consensus.cc:1238] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Rejecting Update request from peer 6ddeb67cfc6e4f5ba3028e51fcdd545c for earlier term 1. Current term is 2. Ops: []
I20250904 13:57:27.947188 10999 consensus_queue.cc:1046] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250904 13:57:28.001978 11066 raft_consensus.cc:491] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.002569 11066 raft_consensus.cc:513] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.004283 11066 leader_election.cc:290] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:28.005131 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8f248691b38b4c76add5969ac40fbdc0" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:28.005407 10918 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8f248691b38b4c76add5969ac40fbdc0" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:28.006703 10735 leader_election.cc:304] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:28.007537 11066 raft_consensus.cc:2747] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:28.022692 11066 raft_consensus.cc:491] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.023361 11066 raft_consensus.cc:513] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.025108 11066 leader_election.cc:290] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:28.026494 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1fd1a3b7dd454961b9ed1dbc3e0af7ef" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:28.026844 10918 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1fd1a3b7dd454961b9ed1dbc3e0af7ef" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:28.028254 10735 leader_election.cc:304] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:28.029124 11066 raft_consensus.cc:2747] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:28.047868 11066 raft_consensus.cc:491] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.048352 11066 raft_consensus.cc:513] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.050002 11066 leader_election.cc:290] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:28.050798 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1cd635db3064f9c94a5dbc6d620db61" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:28.050988 10918 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1cd635db3064f9c94a5dbc6d620db61" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:28.052250 10735 leader_election.cc:304] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:28.052914 11066 raft_consensus.cc:2747] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:28.061175 11066 raft_consensus.cc:491] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.061681 11066 raft_consensus.cc:513] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.063345 11066 leader_election.cc:290] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:28.064088 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7cc86826b94e49289260ba78159669db" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:28.064410 10918 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7cc86826b94e49289260ba78159669db" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:28.065711 10735 leader_election.cc:304] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:28.066432 11066 raft_consensus.cc:2747] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:28.084069 11066 raft_consensus.cc:491] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 926518f7b9994fc288cbdff9d74323a3)
I20250904 13:57:28.084566 11066 raft_consensus.cc:513] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.086150 11066 leader_election.cc:290] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:28.087152 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "27b15f5a5a6f481cbb1af6df1053357a" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:28.087209 10918 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "27b15f5a5a6f481cbb1af6df1053357a" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:28.088721 10735 leader_election.cc:304] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:28.089392 11066 raft_consensus.cc:2747] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:28.107990 10996 raft_consensus.cc:491] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.108505 10996 raft_consensus.cc:513] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.109930 11066 raft_consensus.cc:491] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.110445 10996 leader_election.cc:290] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:28.110464 11066 raft_consensus.cc:513] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.111375 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1d2da9bd6e0405fa033babd924eb743" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:28.111655 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1d2da9bd6e0405fa033babd924eb743" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:28.112679 11066 leader_election.cc:290] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:28.113209 10884 leader_election.cc:304] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:28.113397 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6f08a2bf1d1841a19924894a1af8dcac" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:28.114080 10996 raft_consensus.cc:2747] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:28.114013 10918 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "6f08a2bf1d1841a19924894a1af8dcac" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:28.115355 10735 leader_election.cc:304] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:28.116003 11066 raft_consensus.cc:2747] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:28.144265 11021 raft_consensus.cc:3053] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 LEADER]: Stepping down as leader of term 1
I20250904 13:57:28.144692 11021 raft_consensus.cc:738] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Running, Role: LEADER
I20250904 13:57:28.145388 11021 consensus_queue.cc:260] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } }
I20250904 13:57:28.146647 11021 raft_consensus.cc:3058] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Advancing to term 2
I20250904 13:57:28.171739 10843 raft_consensus.cc:1273] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 2 FOLLOWER]: Refusing update from remote peer 431fe14e8b31422ba906a93b6e5f45be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250904 13:57:28.173008 11066 consensus_queue.cc:1035] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250904 13:57:28.174211 10996 raft_consensus.cc:491] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.174774 10996 raft_consensus.cc:513] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.177193 10996 leader_election.cc:290] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:28.178503 10844 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7cc86826b94e49289260ba78159669db" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:28.178738 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "7cc86826b94e49289260ba78159669db" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:28.180310 10884 leader_election.cc:304] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:28.181190 10996 raft_consensus.cc:2747] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:28.244567 10996 raft_consensus.cc:491] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.245133 10996 raft_consensus.cc:513] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.246897 10996 leader_election.cc:290] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:28.247887 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1cd635db3064f9c94a5dbc6d620db61" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:28.248032 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1cd635db3064f9c94a5dbc6d620db61" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:28.249351 10884 leader_election.cc:304] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:28.250011 11076 raft_consensus.cc:2747] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:28.274637 10918 raft_consensus.cc:1273] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Refusing update from remote peer 431fe14e8b31422ba906a93b6e5f45be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20250904 13:57:28.275894 11066 consensus_queue.cc:1035] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [LEADER]: Connected to new peer: Peer: permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250904 13:57:28.282406 11076 raft_consensus.cc:491] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.282884 11076 raft_consensus.cc:513] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.284823 11076 leader_election.cc:290] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:28.285655 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:28.285739 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:28.286389 10768 raft_consensus.cc:2466] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 926518f7b9994fc288cbdff9d74323a3 in term 1.
I20250904 13:57:28.287678 10884 leader_election.cc:304] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be, 926518f7b9994fc288cbdff9d74323a3; no voters:
I20250904 13:57:28.288456 11076 raft_consensus.cc:2802] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250904 13:57:28.288829 11076 raft_consensus.cc:491] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.289230 11076 raft_consensus.cc:3058] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Advancing to term 2
I20250904 13:57:28.295889 11076 raft_consensus.cc:513] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.298074 11076 leader_election.cc:290] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 election: Requested vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:28.298789 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be"
I20250904 13:57:28.299055 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c"
I20250904 13:57:28.300288 10884 leader_election.cc:304] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:28.300967 11076 raft_consensus.cc:2747] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20250904 13:57:28.336437 10918 raft_consensus.cc:1238] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Rejecting Update request from peer 6ddeb67cfc6e4f5ba3028e51fcdd545c for earlier term 1. Current term is 2. Ops: []
I20250904 13:57:28.337826 11071 consensus_queue.cc:1046] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20250904 13:57:28.375455 11067 raft_consensus.cc:491] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.375963 11067 raft_consensus.cc:513] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.377674 11067 leader_election.cc:290] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:28.378293 11076 raft_consensus.cc:491] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.378743 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1d2da9bd6e0405fa033babd924eb743" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:28.378944 10918 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1d2da9bd6e0405fa033babd924eb743" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:28.378829 11076 raft_consensus.cc:513] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.380307 10735 leader_election.cc:304] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:28.380870 11076 leader_election.cc:290] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:28.381533 11067 raft_consensus.cc:2747] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:28.381592 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8f248691b38b4c76add5969ac40fbdc0" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:28.382562 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "8f248691b38b4c76add5969ac40fbdc0" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:28.383926 10884 leader_election.cc:304] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:28.384573 11076 raft_consensus.cc:2747] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:28.411195 11076 raft_consensus.cc:491] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.411707 11076 raft_consensus.cc:513] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.413314 11076 leader_election.cc:290] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:28.414327 10767 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1fd1a3b7dd454961b9ed1dbc3e0af7ef" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:28.414440 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1fd1a3b7dd454961b9ed1dbc3e0af7ef" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:28.416018 10884 leader_election.cc:304] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:28.416862 11076 raft_consensus.cc:2747] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:28.421442 11076 raft_consensus.cc:491] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.421939 11076 raft_consensus.cc:513] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.424108 11076 leader_election.cc:290] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:28.425127 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5c80f91315f84cf49f62445010a99d83" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:28.425271 10768 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5c80f91315f84cf49f62445010a99d83" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "431fe14e8b31422ba906a93b6e5f45be" is_pre_election: true
I20250904 13:57:28.426915 11067 raft_consensus.cc:491] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.426900 10884 leader_election.cc:304] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:28.427831 11076 raft_consensus.cc:2747] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:28.427582 11067 raft_consensus.cc:513] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.429294 11067 leader_election.cc:290] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:28.430648 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1cd635db3064f9c94a5dbc6d620db61" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:28.430625 10918 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1cd635db3064f9c94a5dbc6d620db61" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:28.432288 10735 leader_election.cc:304] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:28.433043 11067 raft_consensus.cc:2747] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:28.477016 11067 raft_consensus.cc:491] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.477532 11067 raft_consensus.cc:513] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.479147 11067 leader_election.cc:290] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:28.480396 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5c80f91315f84cf49f62445010a99d83" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:28.480401 10918 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5c80f91315f84cf49f62445010a99d83" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:28.482317 10735 leader_election.cc:304] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:28.483136 11067 raft_consensus.cc:2747] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:28.493831 10634 tablet_server.cc:178] TabletServer@127.10.98.129:0 shutting down...
I20250904 13:57:28.520097 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:57:28.520788 10634 tablet_replica.cc:331] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be: stopping tablet replica
I20250904 13:57:28.521586 10634 raft_consensus.cc:2241] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [term 2 LEADER]: Raft consensus shutting down.
W20250904 13:57:28.547894 10810 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20250904 13:57:28.550603 10810 consensus_peers.cc:489] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c -> Peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Couldn't send request to peer 431fe14e8b31422ba906a93b6e5f45be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250904 13:57:28.563793 10999 raft_consensus.cc:3053] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 LEADER]: Stepping down as leader of term 1
I20250904 13:57:28.564190 10999 raft_consensus.cc:738] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ddeb67cfc6e4f5ba3028e51fcdd545c, State: Running, Role: LEADER
I20250904 13:57:28.564993 10999 consensus_queue.cc:260] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.565989 10999 raft_consensus.cc:3058] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Advancing to term 2
W20250904 13:57:28.588897 10810 consensus_peers.cc:489] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c -> Peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Couldn't send request to peer 431fe14e8b31422ba906a93b6e5f45be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250904 13:57:28.591670 10975 raft_consensus.cc:491] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.592144 10975 raft_consensus.cc:513] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.593832 10975 leader_election.cc:290] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:28.594738 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:28.594945 10916 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:28.595336 10843 raft_consensus.cc:2466] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 431fe14e8b31422ba906a93b6e5f45be in term 2.
I20250904 13:57:28.595607 10916 raft_consensus.cc:2391] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 431fe14e8b31422ba906a93b6e5f45be in current term 2: Already voted for candidate 926518f7b9994fc288cbdff9d74323a3 in this term.
I20250904 13:57:28.596273 10735 leader_election.cc:304] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c; no voters:
I20250904 13:57:28.596987 10975 raft_consensus.cc:2802] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election won for term 2
I20250904 13:57:28.597352 10975 raft_consensus.cc:491] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.597683 10975 raft_consensus.cc:3058] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Advancing to term 2
I20250904 13:57:28.602495 10975 raft_consensus.cc:513] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.604380 10975 leader_election.cc:290] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 election: Requested vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:28.605521 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c"
I20250904 13:57:28.605928 10916 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3"
I20250904 13:57:28.607326 10916 raft_consensus.cc:2391] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 431fe14e8b31422ba906a93b6e5f45be in current term 2: Already voted for candidate 926518f7b9994fc288cbdff9d74323a3 in this term.
I20250904 13:57:28.612223 10843 raft_consensus.cc:2466] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 431fe14e8b31422ba906a93b6e5f45be in term 2.
I20250904 13:57:28.613521 10735 leader_election.cc:304] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c; no voters: 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:28.614291 10975 raft_consensus.cc:2802] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 2 FOLLOWER]: Leader election won for term 2
I20250904 13:57:28.614727 10975 raft_consensus.cc:695] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 2 LEADER]: Becoming Leader. State: Replica: 431fe14e8b31422ba906a93b6e5f45be, State: Running, Role: LEADER
I20250904 13:57:28.615458 10975 consensus_queue.cc:237] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.630816 10975 raft_consensus.cc:491] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.631322 10975 raft_consensus.cc:513] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.632887 10634 raft_consensus.cc:2270] T 8718fc5fdf154903a9039ed2c48a85b7 P 431fe14e8b31422ba906a93b6e5f45be [term 2 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:28.633069 10975 leader_election.cc:290] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131:32881)
I20250904 13:57:28.634279 10916 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1fd1a3b7dd454961b9ed1dbc3e0af7ef" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "926518f7b9994fc288cbdff9d74323a3" is_pre_election: true
I20250904 13:57:28.634148 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1fd1a3b7dd454961b9ed1dbc3e0af7ef" candidate_uuid: "431fe14e8b31422ba906a93b6e5f45be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:28.635931 10735 leader_election.cc:304] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 431fe14e8b31422ba906a93b6e5f45be; no voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3
I20250904 13:57:28.636613 10975 raft_consensus.cc:2747] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:28.636842 10634 tablet_replica.cc:331] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be: stopping tablet replica
I20250904 13:57:28.637466 10634 raft_consensus.cc:2241] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:28.637851 10634 raft_consensus.cc:2270] T 7cc86826b94e49289260ba78159669db P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:28.639467 10634 tablet_replica.cc:331] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be: stopping tablet replica
I20250904 13:57:28.639993 10634 raft_consensus.cc:2241] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:28.640429 10634 raft_consensus.cc:2270] T 5c80f91315f84cf49f62445010a99d83 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:28.641902 10634 tablet_replica.cc:331] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be: stopping tablet replica
I20250904 13:57:28.642583 10634 raft_consensus.cc:2241] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:28.642993 10634 raft_consensus.cc:2270] T a1d2da9bd6e0405fa033babd924eb743 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:28.644435 10634 tablet_replica.cc:331] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be: stopping tablet replica
I20250904 13:57:28.644892 10634 raft_consensus.cc:2241] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:28.645210 10634 raft_consensus.cc:2270] T a1cd635db3064f9c94a5dbc6d620db61 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:28.646670 10634 tablet_replica.cc:331] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be: stopping tablet replica
I20250904 13:57:28.647136 10634 raft_consensus.cc:2241] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:28.647457 10634 raft_consensus.cc:2270] T 6f08a2bf1d1841a19924894a1af8dcac P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:28.648901 10634 tablet_replica.cc:331] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be: stopping tablet replica
I20250904 13:57:28.649363 10634 raft_consensus.cc:2241] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:28.649758 10634 raft_consensus.cc:2270] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:28.651482 10634 tablet_replica.cc:331] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be: stopping tablet replica
I20250904 13:57:28.652030 10634 raft_consensus.cc:2241] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 2 LEADER]: Raft consensus shutting down.
I20250904 13:57:28.652730 10634 pending_rounds.cc:70] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be: Trying to abort 1 pending ops.
I20250904 13:57:28.652973 10634 pending_rounds.cc:77] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7196648442336919552 op_type: NO_OP noop_request { }
I20250904 13:57:28.653298 10634 raft_consensus.cc:2887] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
W20250904 13:57:28.653475 10810 consensus_peers.cc:489] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c -> Peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Couldn't send request to peer 431fe14e8b31422ba906a93b6e5f45be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250904 13:57:28.653578 10634 raft_consensus.cc:2270] T bb8182b451f946b78811668ce1d0ce37 P 431fe14e8b31422ba906a93b6e5f45be [term 2 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:28.655488 10634 tablet_replica.cc:331] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be: stopping tablet replica
I20250904 13:57:28.655973 10634 raft_consensus.cc:2241] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:28.656342 10634 raft_consensus.cc:2270] T 27b15f5a5a6f481cbb1af6df1053357a P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:28.657788 10634 tablet_replica.cc:331] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be: stopping tablet replica
I20250904 13:57:28.658265 10634 raft_consensus.cc:2241] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:28.658643 10634 raft_consensus.cc:2270] T 8f248691b38b4c76add5969ac40fbdc0 P 431fe14e8b31422ba906a93b6e5f45be [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:28.686023 11076 raft_consensus.cc:491] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:28.686733 11076 raft_consensus.cc:513] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.689468 11076 leader_election.cc:290] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:28.730439 11076 raft_consensus.cc:491] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.730962 11076 raft_consensus.cc:513] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.732735 11076 leader_election.cc:290] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:28.746831 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:28.747671 10843 raft_consensus.cc:2466] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 926518f7b9994fc288cbdff9d74323a3 in term 2.
W20250904 13:57:28.753049 10810 consensus_peers.cc:489] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c -> Peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Couldn't send request to peer 431fe14e8b31422ba906a93b6e5f45be. Status: Network error: Client connection negotiation failed: client connection to 127.10.98.129:42753: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250904 13:57:28.753597 10810 consensus_peers.cc:489] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c -> Peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Couldn't send request to peer 431fe14e8b31422ba906a93b6e5f45be. Status: Network error: Client connection negotiation failed: client connection to 127.10.98.129:42753: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 13:57:28.753993 10634 tablet_server.cc:195] TabletServer@127.10.98.129:0 shutdown complete.
I20250904 13:57:28.754793 10884 leader_election.cc:304] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3; no voters:
I20250904 13:57:28.755062 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a1d2da9bd6e0405fa033babd924eb743" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" is_pre_election: true
I20250904 13:57:28.755714 10996 raft_consensus.cc:2802] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20250904 13:57:28.756078 10996 raft_consensus.cc:491] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:57:28.756418 10996 raft_consensus.cc:3058] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Advancing to term 3
W20250904 13:57:28.761487 10884 leader_election.cc:336] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Network error: Client connection negotiation failed: client connection to 127.10.98.129:42753: connect: Connection refused (error 111)
W20250904 13:57:28.762318 10884 consensus_peers.cc:489] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 -> Peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Couldn't send request to peer 431fe14e8b31422ba906a93b6e5f45be. Status: Network error: Client connection negotiation failed: client connection to 127.10.98.129:42753: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250904 13:57:28.762737 10884 leader_election.cc:336] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Network error: Client connection negotiation failed: client connection to 127.10.98.129:42753: connect: Connection refused (error 111)
I20250904 13:57:28.766981 10884 leader_election.cc:304] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:28.767746 11076 raft_consensus.cc:2747] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:28.765389 10996 raft_consensus.cc:513] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
W20250904 13:57:28.769728 10810 consensus_peers.cc:489] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c -> Peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Couldn't send request to peer 431fe14e8b31422ba906a93b6e5f45be. Status: Network error: Client connection negotiation failed: client connection to 127.10.98.129:42753: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 13:57:28.770651 10996 leader_election.cc:290] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 3 election: Requested vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:28.772480 10843 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "bb8182b451f946b78811668ce1d0ce37" candidate_uuid: "926518f7b9994fc288cbdff9d74323a3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c"
I20250904 13:57:28.773120 10843 raft_consensus.cc:3058] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 2 FOLLOWER]: Advancing to term 3
W20250904 13:57:28.774941 10884 leader_election.cc:336] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Network error: Client connection negotiation failed: client connection to 127.10.98.129:42753: connect: Connection refused (error 111)
I20250904 13:57:28.778103 10634 tablet_server.cc:178] TabletServer@127.10.98.130:0 shutting down...
W20250904 13:57:28.778048 10810 consensus_peers.cc:489] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c -> Peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Couldn't send request to peer 431fe14e8b31422ba906a93b6e5f45be. Status: Network error: Client connection negotiation failed: client connection to 127.10.98.129:42753: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 13:57:28.780988 10843 raft_consensus.cc:2466] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 926518f7b9994fc288cbdff9d74323a3 in term 3.
I20250904 13:57:28.782414 10884 leader_election.cc:304] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ddeb67cfc6e4f5ba3028e51fcdd545c, 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be
I20250904 13:57:28.783313 10996 raft_consensus.cc:2802] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 3 FOLLOWER]: Leader election won for term 3
I20250904 13:57:28.783741 10996 raft_consensus.cc:695] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 3 LEADER]: Becoming Leader. State: Replica: 926518f7b9994fc288cbdff9d74323a3, State: Running, Role: LEADER
I20250904 13:57:28.784480 10996 consensus_queue.cc:237] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.798502 10665 catalog_manager.cc:5582] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 reported cstate change: term changed from 1 to 3, leader changed from 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130) to 926518f7b9994fc288cbdff9d74323a3 (127.10.98.131). New cstate: current_term: 3 leader_uuid: "926518f7b9994fc288cbdff9d74323a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } health_report { overall_health: HEALTHY } } }
I20250904 13:57:28.806665 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:57:28.807471 10634 tablet_replica.cc:331] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: stopping tablet replica
I20250904 13:57:28.808389 10634 raft_consensus.cc:2241] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 2 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:28.808837 10634 raft_consensus.cc:2270] T 8718fc5fdf154903a9039ed2c48a85b7 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 2 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:28.810775 10634 tablet_replica.cc:331] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c: stopping tablet replica
I20250904 13:57:28.811278 10634 raft_consensus.cc:2241] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:57:28.811956 10634 raft_consensus.cc:2270] T 7cc86826b94e49289260ba78159669db P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:28.813540 10634 tablet_replica.cc:331] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: stopping tablet replica
I20250904 13:57:28.814011 10634 raft_consensus.cc:2241] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:57:28.814785 10634 raft_consensus.cc:2270] T 5c80f91315f84cf49f62445010a99d83 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:28.816263 10634 tablet_replica.cc:331] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c: stopping tablet replica
I20250904 13:57:28.816754 10634 raft_consensus.cc:2241] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:57:28.817404 10634 raft_consensus.cc:2270] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:28.817873 10996 raft_consensus.cc:491] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 431fe14e8b31422ba906a93b6e5f45be)
I20250904 13:57:28.818385 10996 raft_consensus.cc:513] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } }
I20250904 13:57:28.819306 10634 tablet_replica.cc:331] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: stopping tablet replica
I20250904 13:57:28.819865 10634 raft_consensus.cc:2241] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 3 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:28.820117 10996 leader_election.cc:290] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753)
I20250904 13:57:28.820254 10634 raft_consensus.cc:2270] T bb8182b451f946b78811668ce1d0ce37 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 3 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:28.822583 10634 tablet_replica.cc:331] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c: stopping tablet replica
W20250904 13:57:28.822808 10884 leader_election.cc:336] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20250904 13:57:28.823313 10634 raft_consensus.cc:2241] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:57:28.824196 10634 raft_consensus.cc:2270] T 6f08a2bf1d1841a19924894a1af8dcac P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Raft consensus is shut down!
W20250904 13:57:28.824680 10884 leader_election.cc:336] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Network error: Client connection negotiation failed: client connection to 127.10.98.129:42753: connect: Connection refused (error 111)
I20250904 13:57:28.825006 10884 leader_election.cc:304] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:28.825570 10996 raft_consensus.cc:2747] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250904 13:57:28.825980 10634 tablet_replica.cc:331] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: stopping tablet replica
I20250904 13:57:28.826526 10634 raft_consensus.cc:2241] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:57:28.827224 10634 raft_consensus.cc:2270] T a1d2da9bd6e0405fa033babd924eb743 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:28.828824 10634 tablet_replica.cc:331] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: stopping tablet replica
I20250904 13:57:28.829296 10634 raft_consensus.cc:2241] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:57:28.829968 10634 raft_consensus.cc:2270] T a1cd635db3064f9c94a5dbc6d620db61 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:28.831512 10634 tablet_replica.cc:331] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c: stopping tablet replica
I20250904 13:57:28.831959 10634 raft_consensus.cc:2241] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:57:28.832602 10634 raft_consensus.cc:2270] T 8f248691b38b4c76add5969ac40fbdc0 P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:28.834055 10634 tablet_replica.cc:331] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c: stopping tablet replica
I20250904 13:57:28.834527 10634 raft_consensus.cc:2241] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:28.834874 10634 raft_consensus.cc:2270] T 27b15f5a5a6f481cbb1af6df1053357a P 6ddeb67cfc6e4f5ba3028e51fcdd545c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:28.862643 10634 tablet_server.cc:195] TabletServer@127.10.98.130:0 shutdown complete.
I20250904 13:57:28.877799 10634 tablet_server.cc:178] TabletServer@127.10.98.131:0 shutting down...
I20250904 13:57:28.926515 10996 raft_consensus.cc:491] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:28.927059 10996 raft_consensus.cc:513] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:28.928794 10996 leader_election.cc:290] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
W20250904 13:57:28.932435 10884 leader_election.cc:336] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Network error: Client connection negotiation failed: client connection to 127.10.98.129:42753: connect: Connection refused (error 111)
W20250904 13:57:28.933184 10884 leader_election.cc:336] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863): Network error: Client connection negotiation failed: client connection to 127.10.98.130:39863: connect: Connection refused (error 111)
I20250904 13:57:28.933480 10884 leader_election.cc:304] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:28.933993 10996 raft_consensus.cc:2747] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250904 13:57:29.020118 10884 consensus_peers.cc:489] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 -> Peer 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863): Couldn't send request to peer 6ddeb67cfc6e4f5ba3028e51fcdd545c. Status: Network error: Client connection negotiation failed: client connection to 127.10.98.130:39863: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 13:57:29.057502 10996 raft_consensus.cc:491] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:29.058097 10996 raft_consensus.cc:513] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:29.058969 11076 raft_consensus.cc:491] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:29.059455 11076 raft_consensus.cc:513] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:29.060117 10996 leader_election.cc:290] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:29.060609 10996 raft_consensus.cc:491] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:29.061049 10996 raft_consensus.cc:513] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:29.061410 11076 leader_election.cc:290] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:29.063165 10996 leader_election.cc:290] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
W20250904 13:57:29.065073 10884 consensus_peers.cc:489] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 -> Peer 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863): Couldn't send request to peer 6ddeb67cfc6e4f5ba3028e51fcdd545c. Status: Network error: Client connection negotiation failed: client connection to 127.10.98.130:39863: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 13:57:29.065261 10996 raft_consensus.cc:491] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
W20250904 13:57:29.065472 10884 leader_election.cc:336] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863): Network error: Client connection negotiation failed: client connection to 127.10.98.130:39863: connect: Connection refused (error 111)
I20250904 13:57:29.065759 10996 raft_consensus.cc:513] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
W20250904 13:57:29.066387 10884 leader_election.cc:336] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Network error: Client connection negotiation failed: client connection to 127.10.98.129:42753: connect: Connection refused (error 111)
I20250904 13:57:29.066798 10884 leader_election.cc:304] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:29.066797 11076 raft_consensus.cc:491] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6ddeb67cfc6e4f5ba3028e51fcdd545c)
I20250904 13:57:29.067612 10996 leader_election.cc:290] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
I20250904 13:57:29.067625 11089 raft_consensus.cc:2747] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250904 13:57:29.067771 10884 leader_election.cc:336] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Network error: Client connection negotiation failed: client connection to 127.10.98.129:42753: connect: Connection refused (error 111)
I20250904 13:57:29.067543 11076 raft_consensus.cc:513] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } }
I20250904 13:57:29.069729 11076 leader_election.cc:290] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753), 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863)
W20250904 13:57:29.071007 10884 leader_election.cc:336] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863): Network error: Client connection negotiation failed: client connection to 127.10.98.130:39863: connect: Connection refused (error 111)
I20250904 13:57:29.071424 10884 leader_election.cc:304] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:29.071944 11076 raft_consensus.cc:2747] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250904 13:57:29.072000 10884 leader_election.cc:336] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863): Network error: Client connection negotiation failed: client connection to 127.10.98.130:39863: connect: Connection refused (error 111)
W20250904 13:57:29.072769 10884 leader_election.cc:336] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Network error: Client connection negotiation failed: client connection to 127.10.98.129:42753: connect: Connection refused (error 111)
I20250904 13:57:29.073140 10884 leader_election.cc:304] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:29.073650 11076 raft_consensus.cc:2747] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250904 13:57:29.075367 10884 leader_election.cc:336] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863): Network error: Client connection negotiation failed: client connection to 127.10.98.130:39863: connect: Connection refused (error 111)
W20250904 13:57:29.075841 10884 leader_election.cc:336] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863): Network error: Client connection negotiation failed: client connection to 127.10.98.130:39863: connect: Connection refused (error 111)
W20250904 13:57:29.076283 10884 leader_election.cc:336] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Network error: Client connection negotiation failed: client connection to 127.10.98.129:42753: connect: Connection refused (error 111)
I20250904 13:57:29.076515 10884 leader_election.cc:304] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:29.076982 11076 raft_consensus.cc:2747] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20250904 13:57:29.077029 10884 leader_election.cc:336] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Network error: Client connection negotiation failed: client connection to 127.10.98.129:42753: connect: Connection refused (error 111)
I20250904 13:57:29.077481 10884 leader_election.cc:304] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:29.077989 11076 raft_consensus.cc:2747] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20250904 13:57:29.084671 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:57:29.085254 10634 tablet_replica.cc:331] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3: stopping tablet replica
I20250904 13:57:29.085831 10634 raft_consensus.cc:2241] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:29.086272 10634 raft_consensus.cc:2270] T a1cd635db3064f9c94a5dbc6d620db61 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:29.087857 10634 tablet_replica.cc:331] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3: stopping tablet replica
I20250904 13:57:29.088374 10634 raft_consensus.cc:2241] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:29.088755 10634 raft_consensus.cc:2270] T a1d2da9bd6e0405fa033babd924eb743 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:29.090350 10634 tablet_replica.cc:331] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3: stopping tablet replica
I20250904 13:57:29.090862 10634 raft_consensus.cc:2241] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:29.091229 10634 raft_consensus.cc:2270] T 8f248691b38b4c76add5969ac40fbdc0 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:29.092737 10634 tablet_replica.cc:331] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3: stopping tablet replica
I20250904 13:57:29.093221 10634 raft_consensus.cc:2241] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [term 1 LEADER]: Raft consensus shutting down.
W20250904 13:57:29.159607 10884 consensus_peers.cc:489] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 -> Peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Couldn't send request to peer 431fe14e8b31422ba906a93b6e5f45be. Status: Network error: Client connection negotiation failed: client connection to 127.10.98.129:42753: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 13:57:29.175256 11076 raft_consensus.cc:491] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 431fe14e8b31422ba906a93b6e5f45be)
I20250904 13:57:29.175822 11076 raft_consensus.cc:513] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "926518f7b9994fc288cbdff9d74323a3" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 32881 } } peers { permanent_uuid: "6ddeb67cfc6e4f5ba3028e51fcdd545c" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 39863 } } peers { permanent_uuid: "431fe14e8b31422ba906a93b6e5f45be" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42753 } }
I20250904 13:57:29.177667 11076 leader_election.cc:290] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863), 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753)
W20250904 13:57:29.181095 10884 leader_election.cc:336] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6ddeb67cfc6e4f5ba3028e51fcdd545c (127.10.98.130:39863): Network error: Client connection negotiation failed: client connection to 127.10.98.130:39863: connect: Connection refused (error 111)
W20250904 13:57:29.181586 10884 leader_election.cc:336] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 431fe14e8b31422ba906a93b6e5f45be (127.10.98.129:42753): Network error: Client connection negotiation failed: client connection to 127.10.98.129:42753: connect: Connection refused (error 111)
I20250904 13:57:29.181825 10884 leader_election.cc:304] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 926518f7b9994fc288cbdff9d74323a3; no voters: 431fe14e8b31422ba906a93b6e5f45be, 6ddeb67cfc6e4f5ba3028e51fcdd545c
I20250904 13:57:29.182387 11076 raft_consensus.cc:2747] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20250904 13:57:29.215065 10634 raft_consensus.cc:2270] T 27b15f5a5a6f481cbb1af6df1053357a P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:29.216838 10634 tablet_replica.cc:331] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3: stopping tablet replica
I20250904 13:57:29.217350 10634 raft_consensus.cc:2241] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:29.217720 10634 raft_consensus.cc:2270] T 5c80f91315f84cf49f62445010a99d83 P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:29.219288 10634 tablet_replica.cc:331] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3: stopping tablet replica
I20250904 13:57:29.219767 10634 raft_consensus.cc:2241] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 3 LEADER]: Raft consensus shutting down.
I20250904 13:57:29.220443 10634 pending_rounds.cc:70] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3: Trying to abort 1 pending ops.
I20250904 13:57:29.220609 10634 pending_rounds.cc:77] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3: Aborting op as it isn't in flight: id { term: 3 index: 2 } timestamp: 7196648443030368256 op_type: NO_OP noop_request { }
I20250904 13:57:29.220845 10634 raft_consensus.cc:2887] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250904 13:57:29.221065 10634 raft_consensus.cc:2270] T bb8182b451f946b78811668ce1d0ce37 P 926518f7b9994fc288cbdff9d74323a3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:29.223024 10634 tablet_replica.cc:331] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3: stopping tablet replica
I20250904 13:57:29.223572 10634 raft_consensus.cc:2241] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:29.223932 10634 raft_consensus.cc:2270] T 1fd1a3b7dd454961b9ed1dbc3e0af7ef P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:29.225356 10634 tablet_replica.cc:331] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3: stopping tablet replica
I20250904 13:57:29.225776 10634 raft_consensus.cc:2241] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:29.226101 10634 raft_consensus.cc:2270] T 6f08a2bf1d1841a19924894a1af8dcac P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:29.227589 10634 tablet_replica.cc:331] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3: stopping tablet replica
I20250904 13:57:29.228010 10634 raft_consensus.cc:2241] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:29.228394 10634 raft_consensus.cc:2270] T 7cc86826b94e49289260ba78159669db P 926518f7b9994fc288cbdff9d74323a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:29.229779 10634 tablet_replica.cc:331] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3: stopping tablet replica
I20250904 13:57:29.230247 10634 raft_consensus.cc:2241] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:29.230588 10634 raft_consensus.cc:2270] T 8718fc5fdf154903a9039ed2c48a85b7 P 926518f7b9994fc288cbdff9d74323a3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:29.257797 10634 tablet_server.cc:195] TabletServer@127.10.98.131:0 shutdown complete.
I20250904 13:57:29.272904 10634 master.cc:561] Master@127.10.98.190:34657 shutting down...
I20250904 13:57:29.383821 10721 catalog_manager.cc:797] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Closing
I20250904 13:57:29.407708 10634 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:57:29.408298 10634 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:29.408634 10634 tablet_replica.cc:331] T 00000000000000000000000000000000 P 53f6277f96774ff8a2fbe3258bc0c659: stopping tablet replica
I20250904 13:57:29.427193 10634 master.cc:583] Master@127.10.98.190:34657 shutdown complete.
[ OK ] TServerQuiescingITest.TestQuiescingServerDoesntTriggerElections (7550 ms)
[ RUN ] TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership
I20250904 13:57:29.462703 10634 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.98.190:45989
I20250904 13:57:29.463894 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:29.468665 11091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:29.469412 11092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:29.470367 11094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:29.471588 10634 server_base.cc:1047] running on GCE node
I20250904 13:57:29.472362 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:29.472561 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:29.472724 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994249472704 us; error 0 us; skew 500 ppm
I20250904 13:57:29.473223 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:29.475801 10634 webserver.cc:480] Webserver started at http://127.10.98.190:37009/ using document root <none> and password file <none>
I20250904 13:57:29.476298 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:29.476485 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:29.476763 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:29.477854 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership.1756994241738102-10634-0/minicluster-data/master-0-root/instance:
uuid: "fe1bfb3ab71942b68a7217600ae02ccc"
format_stamp: "Formatted at 2025-09-04 13:57:29 on dist-test-slave-jx76"
I20250904 13:57:29.482491 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20250904 13:57:29.485927 11099 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:29.486779 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250904 13:57:29.487064 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership.1756994241738102-10634-0/minicluster-data/master-0-root
uuid: "fe1bfb3ab71942b68a7217600ae02ccc"
format_stamp: "Formatted at 2025-09-04 13:57:29 on dist-test-slave-jx76"
I20250904 13:57:29.487353 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership.1756994241738102-10634-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership.1756994241738102-10634-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership.1756994241738102-10634-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:29.503201 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:29.504263 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:29.540827 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.190:45989
I20250904 13:57:29.540935 11150 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.190:45989 every 8 connection(s)
I20250904 13:57:29.544814 11151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:29.554811 11151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc: Bootstrap starting.
I20250904 13:57:29.559049 11151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:29.563134 11151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc: No bootstrap required, opened a new log
I20250904 13:57:29.565006 11151 raft_consensus.cc:357] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe1bfb3ab71942b68a7217600ae02ccc" member_type: VOTER }
I20250904 13:57:29.565392 11151 raft_consensus.cc:383] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:29.565582 11151 raft_consensus.cc:738] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe1bfb3ab71942b68a7217600ae02ccc, State: Initialized, Role: FOLLOWER
I20250904 13:57:29.566097 11151 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe1bfb3ab71942b68a7217600ae02ccc" member_type: VOTER }
I20250904 13:57:29.566558 11151 raft_consensus.cc:397] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:57:29.566747 11151 raft_consensus.cc:491] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250904 13:57:29.566952 11151 raft_consensus.cc:3058] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:29.571182 11151 raft_consensus.cc:513] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe1bfb3ab71942b68a7217600ae02ccc" member_type: VOTER }
I20250904 13:57:29.571708 11151 leader_election.cc:304] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe1bfb3ab71942b68a7217600ae02ccc; no voters:
I20250904 13:57:29.572926 11151 leader_election.cc:290] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:57:29.573256 11154 raft_consensus.cc:2802] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:29.574663 11154 raft_consensus.cc:695] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc [term 1 LEADER]: Becoming Leader. State: Replica: fe1bfb3ab71942b68a7217600ae02ccc, State: Running, Role: LEADER
I20250904 13:57:29.575397 11154 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe1bfb3ab71942b68a7217600ae02ccc" member_type: VOTER }
I20250904 13:57:29.575954 11151 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc [sys.catalog]: configured and running, proceeding with master startup.
I20250904 13:57:29.580181 11155 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe1bfb3ab71942b68a7217600ae02ccc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe1bfb3ab71942b68a7217600ae02ccc" member_type: VOTER } }
I20250904 13:57:29.580312 11156 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe1bfb3ab71942b68a7217600ae02ccc. Latest consensus state: current_term: 1 leader_uuid: "fe1bfb3ab71942b68a7217600ae02ccc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe1bfb3ab71942b68a7217600ae02ccc" member_type: VOTER } }
I20250904 13:57:29.581254 11155 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc [sys.catalog]: This master's current role is: LEADER
I20250904 13:57:29.581255 11156 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc [sys.catalog]: This master's current role is: LEADER
I20250904 13:57:29.590104 11162 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 13:57:29.597524 11162 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 13:57:29.598618 10634 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 13:57:29.606537 11162 catalog_manager.cc:1349] Generated new cluster ID: 2912bbf2e6e74584aed794469178fde2
I20250904 13:57:29.606859 11162 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 13:57:29.633149 11162 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 13:57:29.634480 11162 catalog_manager.cc:1506] Loading token signing keys...
I20250904 13:57:29.653307 11162 catalog_manager.cc:5955] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc: Generated new TSK 0
I20250904 13:57:29.653946 11162 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 13:57:29.665165 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:29.670783 11172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:29.672076 11173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:29.673909 10634 server_base.cc:1047] running on GCE node
W20250904 13:57:29.674332 11175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:29.675209 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:29.675444 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:29.675597 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994249675582 us; error 0 us; skew 500 ppm
I20250904 13:57:29.676193 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:29.679996 10634 webserver.cc:480] Webserver started at http://127.10.98.129:33005/ using document root <none> and password file <none>
I20250904 13:57:29.680534 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:29.680722 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:29.680989 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:29.682200 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership.1756994241738102-10634-0/minicluster-data/ts-0-root/instance:
uuid: "b9b258a4ef54494c990ff16dcac4002e"
format_stamp: "Formatted at 2025-09-04 13:57:29 on dist-test-slave-jx76"
I20250904 13:57:29.686757 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20250904 13:57:29.690255 11180 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:29.691008 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250904 13:57:29.691304 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership.1756994241738102-10634-0/minicluster-data/ts-0-root
uuid: "b9b258a4ef54494c990ff16dcac4002e"
format_stamp: "Formatted at 2025-09-04 13:57:29 on dist-test-slave-jx76"
I20250904 13:57:29.691591 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership.1756994241738102-10634-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership.1756994241738102-10634-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership.1756994241738102-10634-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:29.711601 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:29.712798 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:29.714296 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:57:29.717070 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:57:29.717384 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:29.717748 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:57:29.717980 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:29.761008 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.129:40591
I20250904 13:57:29.761092 11242 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.129:40591 every 8 connection(s)
I20250904 13:57:29.765988 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:29.773288 11247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:29.776006 11248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:29.777510 10634 server_base.cc:1047] running on GCE node
W20250904 13:57:29.779249 11250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:29.779464 11243 heartbeater.cc:344] Connected to a master server at 127.10.98.190:45989
I20250904 13:57:29.779911 11243 heartbeater.cc:461] Registering TS with master...
I20250904 13:57:29.780148 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:29.780361 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:29.780531 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994249780511 us; error 0 us; skew 500 ppm
I20250904 13:57:29.780650 11243 heartbeater.cc:507] Master 127.10.98.190:45989 requested a full tablet report, sending...
I20250904 13:57:29.781095 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:29.782815 11116 ts_manager.cc:194] Registered new tserver with Master: b9b258a4ef54494c990ff16dcac4002e (127.10.98.129:40591)
I20250904 13:57:29.784476 11116 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48842
I20250904 13:57:29.786383 10634 webserver.cc:480] Webserver started at http://127.10.98.130:37995/ using document root <none> and password file <none>
I20250904 13:57:29.786950 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:29.787173 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:29.787474 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:29.788789 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership.1756994241738102-10634-0/minicluster-data/ts-1-root/instance:
uuid: "44ef7b3bceef4c6d812df790749cde1f"
format_stamp: "Formatted at 2025-09-04 13:57:29 on dist-test-slave-jx76"
I20250904 13:57:29.793440 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20250904 13:57:29.797163 11255 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:29.798041 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250904 13:57:29.798463 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership.1756994241738102-10634-0/minicluster-data/ts-1-root
uuid: "44ef7b3bceef4c6d812df790749cde1f"
format_stamp: "Formatted at 2025-09-04 13:57:29 on dist-test-slave-jx76"
I20250904 13:57:29.798848 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership.1756994241738102-10634-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership.1756994241738102-10634-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership.1756994241738102-10634-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:29.813489 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:29.814625 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:29.816045 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:57:29.818481 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:57:29.818662 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:29.818907 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:57:29.819052 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:29.857092 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.130:45759
I20250904 13:57:29.857193 11317 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.130:45759 every 8 connection(s)
I20250904 13:57:29.862073 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:29.868459 11321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:29.870276 11322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:29.871475 11324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:29.873059 10634 server_base.cc:1047] running on GCE node
I20250904 13:57:29.873733 11318 heartbeater.cc:344] Connected to a master server at 127.10.98.190:45989
I20250904 13:57:29.874069 11318 heartbeater.cc:461] Registering TS with master...
I20250904 13:57:29.874099 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:29.874459 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:29.874619 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994249874602 us; error 0 us; skew 500 ppm
I20250904 13:57:29.874950 11318 heartbeater.cc:507] Master 127.10.98.190:45989 requested a full tablet report, sending...
I20250904 13:57:29.875134 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:29.877218 11116 ts_manager.cc:194] Registered new tserver with Master: 44ef7b3bceef4c6d812df790749cde1f (127.10.98.130:45759)
I20250904 13:57:29.878098 10634 webserver.cc:480] Webserver started at http://127.10.98.131:40981/ using document root <none> and password file <none>
I20250904 13:57:29.878808 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:29.879046 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:29.879210 11116 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48854
I20250904 13:57:29.879382 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:29.880916 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership.1756994241738102-10634-0/minicluster-data/ts-2-root/instance:
uuid: "dd40934f0c3f4e34ad6afaf397700fa4"
format_stamp: "Formatted at 2025-09-04 13:57:29 on dist-test-slave-jx76"
I20250904 13:57:29.885560 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20250904 13:57:29.889082 11329 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:29.889894 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250904 13:57:29.890215 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership.1756994241738102-10634-0/minicluster-data/ts-2-root
uuid: "dd40934f0c3f4e34ad6afaf397700fa4"
format_stamp: "Formatted at 2025-09-04 13:57:29 on dist-test-slave-jx76"
I20250904 13:57:29.890507 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership.1756994241738102-10634-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership.1756994241738102-10634-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership.1756994241738102-10634-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:29.911536 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:29.912711 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:29.914150 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:57:29.916565 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:57:29.916772 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:29.917007 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:57:29.917163 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:29.955296 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.131:43327
I20250904 13:57:29.955382 11391 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.131:43327 every 8 connection(s)
I20250904 13:57:29.968941 11392 heartbeater.cc:344] Connected to a master server at 127.10.98.190:45989
I20250904 13:57:29.969326 11392 heartbeater.cc:461] Registering TS with master...
I20250904 13:57:29.970098 11392 heartbeater.cc:507] Master 127.10.98.190:45989 requested a full tablet report, sending...
I20250904 13:57:29.972115 11116 ts_manager.cc:194] Registered new tserver with Master: dd40934f0c3f4e34ad6afaf397700fa4 (127.10.98.131:43327)
I20250904 13:57:29.972283 10634 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013779258s
I20250904 13:57:29.974032 11116 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48862
I20250904 13:57:30.005846 10634 test_util.cc:276] Using random seed: 438623548
I20250904 13:57:30.028568 11116 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48902:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 13:57:30.030514 11116 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 13:57:30.084550 11357 tablet_service.cc:1468] Processing CreateTablet for tablet 031ad43ad36e4e5082d28a851ac7381f (DEFAULT_TABLE table=test-workload [id=42d8aae02bb14a0d88104f99734dca45]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:57:30.085897 11357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 031ad43ad36e4e5082d28a851ac7381f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:30.089519 11283 tablet_service.cc:1468] Processing CreateTablet for tablet 031ad43ad36e4e5082d28a851ac7381f (DEFAULT_TABLE table=test-workload [id=42d8aae02bb14a0d88104f99734dca45]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:57:30.090997 11283 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 031ad43ad36e4e5082d28a851ac7381f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:30.099157 11208 tablet_service.cc:1468] Processing CreateTablet for tablet 031ad43ad36e4e5082d28a851ac7381f (DEFAULT_TABLE table=test-workload [id=42d8aae02bb14a0d88104f99734dca45]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:57:30.100551 11208 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 031ad43ad36e4e5082d28a851ac7381f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:30.104404 11413 tablet_bootstrap.cc:492] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4: Bootstrap starting.
I20250904 13:57:30.111136 11413 tablet_bootstrap.cc:654] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:30.115271 11414 tablet_bootstrap.cc:492] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f: Bootstrap starting.
I20250904 13:57:30.119020 11413 tablet_bootstrap.cc:492] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4: No bootstrap required, opened a new log
I20250904 13:57:30.119652 11413 ts_tablet_manager.cc:1397] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.001s
I20250904 13:57:30.122416 11414 tablet_bootstrap.cc:654] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:30.122457 11413 raft_consensus.cc:357] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 43327 } } peers { permanent_uuid: "44ef7b3bceef4c6d812df790749cde1f" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 45759 } } peers { permanent_uuid: "b9b258a4ef54494c990ff16dcac4002e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 40591 } }
I20250904 13:57:30.123598 11413 raft_consensus.cc:383] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:30.123991 11413 raft_consensus.cc:738] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd40934f0c3f4e34ad6afaf397700fa4, State: Initialized, Role: FOLLOWER
I20250904 13:57:30.125223 11413 consensus_queue.cc:260] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 43327 } } peers { permanent_uuid: "44ef7b3bceef4c6d812df790749cde1f" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 45759 } } peers { permanent_uuid: "b9b258a4ef54494c990ff16dcac4002e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 40591 } }
I20250904 13:57:30.127321 11416 tablet_bootstrap.cc:492] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e: Bootstrap starting.
I20250904 13:57:30.135305 11413 ts_tablet_manager.cc:1428] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4: Time spent starting tablet: real 0.015s user 0.007s sys 0.008s
I20250904 13:57:30.135555 11416 tablet_bootstrap.cc:654] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:30.136220 11392 heartbeater.cc:499] Master 127.10.98.190:45989 was elected leader, sending a full tablet report...
I20250904 13:57:30.138688 11414 tablet_bootstrap.cc:492] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f: No bootstrap required, opened a new log
I20250904 13:57:30.139233 11414 ts_tablet_manager.cc:1397] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.012s
I20250904 13:57:30.141959 11414 raft_consensus.cc:357] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 43327 } } peers { permanent_uuid: "44ef7b3bceef4c6d812df790749cde1f" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 45759 } } peers { permanent_uuid: "b9b258a4ef54494c990ff16dcac4002e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 40591 } }
I20250904 13:57:30.142788 11414 raft_consensus.cc:383] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:30.143095 11414 raft_consensus.cc:738] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44ef7b3bceef4c6d812df790749cde1f, State: Initialized, Role: FOLLOWER
I20250904 13:57:30.143860 11414 consensus_queue.cc:260] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 43327 } } peers { permanent_uuid: "44ef7b3bceef4c6d812df790749cde1f" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 45759 } } peers { permanent_uuid: "b9b258a4ef54494c990ff16dcac4002e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 40591 } }
I20250904 13:57:30.146366 11420 raft_consensus.cc:491] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:30.146664 11318 heartbeater.cc:499] Master 127.10.98.190:45989 was elected leader, sending a full tablet report...
I20250904 13:57:30.146884 11420 raft_consensus.cc:513] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 43327 } } peers { permanent_uuid: "44ef7b3bceef4c6d812df790749cde1f" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 45759 } } peers { permanent_uuid: "b9b258a4ef54494c990ff16dcac4002e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 40591 } }
I20250904 13:57:30.147500 11414 ts_tablet_manager.cc:1428] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20250904 13:57:30.151398 11416 tablet_bootstrap.cc:492] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e: No bootstrap required, opened a new log
I20250904 13:57:30.152077 11416 ts_tablet_manager.cc:1397] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e: Time spent bootstrapping tablet: real 0.025s user 0.001s sys 0.011s
I20250904 13:57:30.155236 11420 leader_election.cc:290] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd40934f0c3f4e34ad6afaf397700fa4 (127.10.98.131:43327), b9b258a4ef54494c990ff16dcac4002e (127.10.98.129:40591)
I20250904 13:57:30.155488 11416 raft_consensus.cc:357] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 43327 } } peers { permanent_uuid: "44ef7b3bceef4c6d812df790749cde1f" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 45759 } } peers { permanent_uuid: "b9b258a4ef54494c990ff16dcac4002e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 40591 } }
I20250904 13:57:30.156348 11416 raft_consensus.cc:383] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:30.156668 11416 raft_consensus.cc:738] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9b258a4ef54494c990ff16dcac4002e, State: Initialized, Role: FOLLOWER
I20250904 13:57:30.157573 11416 consensus_queue.cc:260] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 43327 } } peers { permanent_uuid: "44ef7b3bceef4c6d812df790749cde1f" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 45759 } } peers { permanent_uuid: "b9b258a4ef54494c990ff16dcac4002e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 40591 } }
I20250904 13:57:30.172251 11218 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "031ad43ad36e4e5082d28a851ac7381f" candidate_uuid: "44ef7b3bceef4c6d812df790749cde1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9b258a4ef54494c990ff16dcac4002e" is_pre_election: true
I20250904 13:57:30.173115 11218 raft_consensus.cc:2466] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44ef7b3bceef4c6d812df790749cde1f in term 0.
I20250904 13:57:30.174161 11367 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "031ad43ad36e4e5082d28a851ac7381f" candidate_uuid: "44ef7b3bceef4c6d812df790749cde1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" is_pre_election: true
I20250904 13:57:30.174607 11259 leader_election.cc:304] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44ef7b3bceef4c6d812df790749cde1f, b9b258a4ef54494c990ff16dcac4002e; no voters:
I20250904 13:57:30.174930 11367 raft_consensus.cc:2466] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44ef7b3bceef4c6d812df790749cde1f in term 0.
I20250904 13:57:30.175375 11424 raft_consensus.cc:491] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:30.175624 11421 raft_consensus.cc:2802] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:57:30.176488 11421 raft_consensus.cc:491] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:57:30.176527 11424 raft_consensus.cc:513] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 43327 } } peers { permanent_uuid: "44ef7b3bceef4c6d812df790749cde1f" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 45759 } } peers { permanent_uuid: "b9b258a4ef54494c990ff16dcac4002e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 40591 } }
I20250904 13:57:30.176105 11243 heartbeater.cc:499] Master 127.10.98.190:45989 was elected leader, sending a full tablet report...
I20250904 13:57:30.176961 11421 raft_consensus.cc:3058] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:30.175616 11416 ts_tablet_manager.cc:1428] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e: Time spent starting tablet: real 0.023s user 0.015s sys 0.001s
I20250904 13:57:30.182631 11424 leader_election.cc:290] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd40934f0c3f4e34ad6afaf397700fa4 (127.10.98.131:43327), 44ef7b3bceef4c6d812df790749cde1f (127.10.98.130:45759)
I20250904 13:57:30.185621 11421 raft_consensus.cc:513] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 43327 } } peers { permanent_uuid: "44ef7b3bceef4c6d812df790749cde1f" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 45759 } } peers { permanent_uuid: "b9b258a4ef54494c990ff16dcac4002e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 40591 } }
I20250904 13:57:30.189678 11367 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "031ad43ad36e4e5082d28a851ac7381f" candidate_uuid: "44ef7b3bceef4c6d812df790749cde1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd40934f0c3f4e34ad6afaf397700fa4"
I20250904 13:57:30.190754 11367 raft_consensus.cc:3058] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:30.194018 11366 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "031ad43ad36e4e5082d28a851ac7381f" candidate_uuid: "b9b258a4ef54494c990ff16dcac4002e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" is_pre_election: true
I20250904 13:57:30.194617 11421 leader_election.cc:290] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [CANDIDATE]: Term 1 election: Requested vote from peers dd40934f0c3f4e34ad6afaf397700fa4 (127.10.98.131:43327), b9b258a4ef54494c990ff16dcac4002e (127.10.98.129:40591)
I20250904 13:57:30.196508 11218 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "031ad43ad36e4e5082d28a851ac7381f" candidate_uuid: "44ef7b3bceef4c6d812df790749cde1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9b258a4ef54494c990ff16dcac4002e"
I20250904 13:57:30.197172 11218 raft_consensus.cc:3058] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:30.198437 11367 raft_consensus.cc:2466] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44ef7b3bceef4c6d812df790749cde1f in term 1.
I20250904 13:57:30.199601 11258 leader_election.cc:304] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44ef7b3bceef4c6d812df790749cde1f, dd40934f0c3f4e34ad6afaf397700fa4; no voters:
I20250904 13:57:30.200601 11293 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "031ad43ad36e4e5082d28a851ac7381f" candidate_uuid: "b9b258a4ef54494c990ff16dcac4002e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ef7b3bceef4c6d812df790749cde1f" is_pre_election: true
I20250904 13:57:30.200593 11421 raft_consensus.cc:2802] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:30.201682 11421 raft_consensus.cc:695] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 1 LEADER]: Becoming Leader. State: Replica: 44ef7b3bceef4c6d812df790749cde1f, State: Running, Role: LEADER
I20250904 13:57:30.202533 11421 consensus_queue.cc:237] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 43327 } } peers { permanent_uuid: "44ef7b3bceef4c6d812df790749cde1f" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 45759 } } peers { permanent_uuid: "b9b258a4ef54494c990ff16dcac4002e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 40591 } }
I20250904 13:57:30.203759 11218 raft_consensus.cc:2466] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44ef7b3bceef4c6d812df790749cde1f in term 1.
I20250904 13:57:30.209167 11183 leader_election.cc:304] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b9b258a4ef54494c990ff16dcac4002e; no voters: 44ef7b3bceef4c6d812df790749cde1f, dd40934f0c3f4e34ad6afaf397700fa4
I20250904 13:57:30.210048 11424 raft_consensus.cc:2747] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20250904 13:57:30.211619 11116 catalog_manager.cc:5582] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 44ef7b3bceef4c6d812df790749cde1f (127.10.98.130). New cstate: current_term: 1 leader_uuid: "44ef7b3bceef4c6d812df790749cde1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 43327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44ef7b3bceef4c6d812df790749cde1f" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 45759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b9b258a4ef54494c990ff16dcac4002e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 40591 } health_report { overall_health: UNKNOWN } } }
I20250904 13:57:30.773293 11427 consensus_queue.cc:1035] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:57:30.781172 11430 raft_consensus.cc:991] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f: : Instructing follower dd40934f0c3f4e34ad6afaf397700fa4 to start an election
I20250904 13:57:30.781941 11427 raft_consensus.cc:1079] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 1 LEADER]: Signalling peer dd40934f0c3f4e34ad6afaf397700fa4 to start an election
I20250904 13:57:30.784504 11366 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "031ad43ad36e4e5082d28a851ac7381f"
dest_uuid: "dd40934f0c3f4e34ad6afaf397700fa4"
from {username='slave'} at 127.0.0.1:54296
I20250904 13:57:30.787520 11366 raft_consensus.cc:491] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250904 13:57:30.787979 11366 raft_consensus.cc:3058] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 1 FOLLOWER]: Advancing to term 2
I20250904 13:57:30.788157 11430 consensus_queue.cc:1035] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9b258a4ef54494c990ff16dcac4002e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 40591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:57:30.789172 11427 raft_consensus.cc:991] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f: : Instructing follower dd40934f0c3f4e34ad6afaf397700fa4 to start an election
I20250904 13:57:30.789611 11427 raft_consensus.cc:1079] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 1 LEADER]: Signalling peer dd40934f0c3f4e34ad6afaf397700fa4 to start an election
I20250904 13:57:30.791352 11367 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "031ad43ad36e4e5082d28a851ac7381f"
dest_uuid: "dd40934f0c3f4e34ad6afaf397700fa4"
from {username='slave'} at 127.0.0.1:54296
I20250904 13:57:30.796314 11366 raft_consensus.cc:513] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 43327 } } peers { permanent_uuid: "44ef7b3bceef4c6d812df790749cde1f" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 45759 } } peers { permanent_uuid: "b9b258a4ef54494c990ff16dcac4002e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 40591 } }
I20250904 13:57:30.797545 11367 raft_consensus.cc:491] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250904 13:57:30.798000 11367 raft_consensus.cc:3058] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 2 FOLLOWER]: Advancing to term 3
I20250904 13:57:30.799034 11427 raft_consensus.cc:991] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f: : Instructing follower b9b258a4ef54494c990ff16dcac4002e to start an election
I20250904 13:57:30.799476 11427 raft_consensus.cc:1079] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 1 LEADER]: Signalling peer b9b258a4ef54494c990ff16dcac4002e to start an election
I20250904 13:57:30.801119 11218 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "031ad43ad36e4e5082d28a851ac7381f"
dest_uuid: "b9b258a4ef54494c990ff16dcac4002e"
from {username='slave'} at 127.0.0.1:53808
I20250904 13:57:30.801720 11218 raft_consensus.cc:491] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250904 13:57:30.802110 11218 raft_consensus.cc:3058] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [term 1 FOLLOWER]: Advancing to term 2
I20250904 13:57:30.805132 11367 raft_consensus.cc:513] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 43327 } } peers { permanent_uuid: "44ef7b3bceef4c6d812df790749cde1f" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 45759 } } peers { permanent_uuid: "b9b258a4ef54494c990ff16dcac4002e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 40591 } }
I20250904 13:57:30.808724 11218 raft_consensus.cc:513] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 43327 } } peers { permanent_uuid: "44ef7b3bceef4c6d812df790749cde1f" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 45759 } } peers { permanent_uuid: "b9b258a4ef54494c990ff16dcac4002e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 40591 } }
I20250904 13:57:30.809221 11366 leader_election.cc:290] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [CANDIDATE]: Term 2 election: Requested vote from peers 44ef7b3bceef4c6d812df790749cde1f (127.10.98.130:45759), b9b258a4ef54494c990ff16dcac4002e (127.10.98.129:40591)
I20250904 13:57:30.811832 11366 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "031ad43ad36e4e5082d28a851ac7381f" candidate_uuid: "b9b258a4ef54494c990ff16dcac4002e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "dd40934f0c3f4e34ad6afaf397700fa4"
I20250904 13:57:30.812443 11293 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "031ad43ad36e4e5082d28a851ac7381f" candidate_uuid: "b9b258a4ef54494c990ff16dcac4002e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "44ef7b3bceef4c6d812df790749cde1f"
I20250904 13:57:30.812966 11366 raft_consensus.cc:2366] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b9b258a4ef54494c990ff16dcac4002e for earlier term 2. Current term is 3.
I20250904 13:57:30.813302 11293 raft_consensus.cc:3053] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 1 LEADER]: Stepping down as leader of term 1
I20250904 13:57:30.813771 11293 raft_consensus.cc:738] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 44ef7b3bceef4c6d812df790749cde1f, State: Running, Role: LEADER
I20250904 13:57:30.814883 11293 consensus_queue.cc:260] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 43327 } } peers { permanent_uuid: "44ef7b3bceef4c6d812df790749cde1f" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 45759 } } peers { permanent_uuid: "b9b258a4ef54494c990ff16dcac4002e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 40591 } }
I20250904 13:57:30.816246 11293 raft_consensus.cc:3058] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 1 FOLLOWER]: Advancing to term 2
I20250904 13:57:30.818589 11183 leader_election.cc:400] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [CANDIDATE]: Term 2 election: Vote denied by peer dd40934f0c3f4e34ad6afaf397700fa4 (127.10.98.131:43327) with higher term. Message: Invalid argument: T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b9b258a4ef54494c990ff16dcac4002e for earlier term 2. Current term is 3.
I20250904 13:57:30.818956 11183 leader_election.cc:403] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20250904 13:57:30.819598 11434 raft_consensus.cc:3058] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [term 2 FOLLOWER]: Advancing to term 3
I20250904 13:57:30.825390 11218 leader_election.cc:290] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [CANDIDATE]: Term 2 election: Requested vote from peers dd40934f0c3f4e34ad6afaf397700fa4 (127.10.98.131:43327), 44ef7b3bceef4c6d812df790749cde1f (127.10.98.130:45759)
I20250904 13:57:30.827980 11293 raft_consensus.cc:2466] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9b258a4ef54494c990ff16dcac4002e in term 2.
I20250904 13:57:30.830463 11434 raft_consensus.cc:2747] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer dd40934f0c3f4e34ad6afaf397700fa4 (127.10.98.131:43327) with higher term. Message: Invalid argument: T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b9b258a4ef54494c990ff16dcac4002e for earlier term 2. Current term is 3.
I20250904 13:57:30.809221 11367 leader_election.cc:290] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [CANDIDATE]: Term 3 election: Requested vote from peers 44ef7b3bceef4c6d812df790749cde1f (127.10.98.130:45759), b9b258a4ef54494c990ff16dcac4002e (127.10.98.129:40591)
I20250904 13:57:30.843200 11293 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "031ad43ad36e4e5082d28a851ac7381f" candidate_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "44ef7b3bceef4c6d812df790749cde1f"
I20250904 13:57:30.843878 11293 raft_consensus.cc:3058] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 2 FOLLOWER]: Advancing to term 3
I20250904 13:57:30.845679 11217 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "031ad43ad36e4e5082d28a851ac7381f" candidate_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b9b258a4ef54494c990ff16dcac4002e"
I20250904 13:57:30.846503 11217 raft_consensus.cc:2366] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd40934f0c3f4e34ad6afaf397700fa4 for earlier term 2. Current term is 3.
I20250904 13:57:30.847797 11217 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "031ad43ad36e4e5082d28a851ac7381f" candidate_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b9b258a4ef54494c990ff16dcac4002e"
I20250904 13:57:30.850605 11292 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "031ad43ad36e4e5082d28a851ac7381f" candidate_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "44ef7b3bceef4c6d812df790749cde1f"
I20250904 13:57:30.853279 11293 raft_consensus.cc:2466] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd40934f0c3f4e34ad6afaf397700fa4 in term 3.
I20250904 13:57:30.854758 11333 leader_election.cc:400] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [CANDIDATE]: Term 2 election: Vote denied by peer b9b258a4ef54494c990ff16dcac4002e (127.10.98.129:40591) with higher term. Message: Invalid argument: T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd40934f0c3f4e34ad6afaf397700fa4 for earlier term 2. Current term is 3.
I20250904 13:57:30.854491 11332 leader_election.cc:304] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44ef7b3bceef4c6d812df790749cde1f, dd40934f0c3f4e34ad6afaf397700fa4; no voters:
I20250904 13:57:30.855151 11333 leader_election.cc:403] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20250904 13:57:30.855726 11428 raft_consensus.cc:2802] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 3 FOLLOWER]: Leader election won for term 3
I20250904 13:57:30.857372 11428 raft_consensus.cc:695] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 3 LEADER]: Becoming Leader. State: Replica: dd40934f0c3f4e34ad6afaf397700fa4, State: Running, Role: LEADER
I20250904 13:57:30.858266 11428 consensus_queue.cc:237] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 43327 } } peers { permanent_uuid: "44ef7b3bceef4c6d812df790749cde1f" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 45759 } } peers { permanent_uuid: "b9b258a4ef54494c990ff16dcac4002e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 40591 } }
I20250904 13:57:30.859052 11217 raft_consensus.cc:2466] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd40934f0c3f4e34ad6afaf397700fa4 in term 3.
I20250904 13:57:30.862273 11444 raft_consensus.cc:2747] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 3 LEADER]: Leader election lost for term 2. Reason: Vote denied by peer b9b258a4ef54494c990ff16dcac4002e (127.10.98.129:40591) with higher term. Message: Invalid argument: T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate dd40934f0c3f4e34ad6afaf397700fa4 for earlier term 2. Current term is 3.
I20250904 13:57:30.868320 11114 catalog_manager.cc:5582] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 reported cstate change: term changed from 1 to 3, leader changed from 44ef7b3bceef4c6d812df790749cde1f (127.10.98.130) to dd40934f0c3f4e34ad6afaf397700fa4 (127.10.98.131). New cstate: current_term: 3 leader_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd40934f0c3f4e34ad6afaf397700fa4" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 43327 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "44ef7b3bceef4c6d812df790749cde1f" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 45759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b9b258a4ef54494c990ff16dcac4002e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 40591 } health_report { overall_health: UNKNOWN } } }
I20250904 13:57:30.869256 10634 tablet_server.cc:178] TabletServer@127.10.98.129:0 shutting down...
I20250904 13:57:30.889048 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:57:30.889680 10634 tablet_replica.cc:331] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e: stopping tablet replica
I20250904 13:57:30.890256 10634 raft_consensus.cc:2241] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [term 3 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:30.890667 10634 raft_consensus.cc:2270] T 031ad43ad36e4e5082d28a851ac7381f P b9b258a4ef54494c990ff16dcac4002e [term 3 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:30.910004 10634 tablet_server.cc:195] TabletServer@127.10.98.129:0 shutdown complete.
I20250904 13:57:30.918442 10634 tablet_server.cc:178] TabletServer@127.10.98.130:0 shutting down...
I20250904 13:57:30.935941 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:57:30.936589 10634 tablet_replica.cc:331] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f: stopping tablet replica
I20250904 13:57:30.937134 10634 raft_consensus.cc:2241] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 3 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:30.937592 10634 raft_consensus.cc:2270] T 031ad43ad36e4e5082d28a851ac7381f P 44ef7b3bceef4c6d812df790749cde1f [term 3 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:30.947021 10634 tablet_server.cc:195] TabletServer@127.10.98.130:0 shutdown complete.
I20250904 13:57:30.955367 10634 tablet_server.cc:178] TabletServer@127.10.98.131:0 shutting down...
I20250904 13:57:30.971522 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:57:30.972101 10634 tablet_replica.cc:331] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4: stopping tablet replica
I20250904 13:57:30.972587 10634 raft_consensus.cc:2241] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 3 LEADER]: Raft consensus shutting down.
I20250904 13:57:30.973176 10634 pending_rounds.cc:70] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4: Trying to abort 1 pending ops.
I20250904 13:57:30.973338 10634 pending_rounds.cc:77] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4: Aborting op as it isn't in flight: id { term: 3 index: 2 } timestamp: 7196648451525156864 op_type: NO_OP noop_request { }
I20250904 13:57:30.973641 10634 raft_consensus.cc:2887] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250904 13:57:30.973912 10634 raft_consensus.cc:2270] T 031ad43ad36e4e5082d28a851ac7381f P dd40934f0c3f4e34ad6afaf397700fa4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:30.993745 10634 tablet_server.cc:195] TabletServer@127.10.98.131:0 shutdown complete.
I20250904 13:57:31.002095 10634 master.cc:561] Master@127.10.98.190:45989 shutting down...
I20250904 13:57:31.017336 10634 raft_consensus.cc:2241] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:57:31.017902 10634 raft_consensus.cc:2270] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:31.018237 10634 tablet_replica.cc:331] T 00000000000000000000000000000000 P fe1bfb3ab71942b68a7217600ae02ccc: stopping tablet replica
I20250904 13:57:31.036254 10634 master.cc:583] Master@127.10.98.190:45989 shutdown complete.
[ OK ] TServerQuiescingITest.TestQuiescingLeaderTransfersLeadership (1597 ms)
[ RUN ] TServerQuiescingITest.TestMajorityQuiescingElectsLeader
I20250904 13:57:31.059782 10634 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.98.190:45057
I20250904 13:57:31.060925 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:31.065831 11448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:31.065944 11449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:31.067409 11451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:31.068943 10634 server_base.cc:1047] running on GCE node
I20250904 13:57:31.069746 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:31.069926 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:31.070089 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994251070071 us; error 0 us; skew 500 ppm
I20250904 13:57:31.070717 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:31.073024 10634 webserver.cc:480] Webserver started at http://127.10.98.190:43215/ using document root <none> and password file <none>
I20250904 13:57:31.073513 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:31.073686 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:31.073962 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:31.075091 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestMajorityQuiescingElectsLeader.1756994241738102-10634-0/minicluster-data/master-0-root/instance:
uuid: "2916f5d8da364a29ad8c7bfb96e06177"
format_stamp: "Formatted at 2025-09-04 13:57:31 on dist-test-slave-jx76"
I20250904 13:57:31.079799 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250904 13:57:31.083367 11456 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:31.084159 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250904 13:57:31.084466 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestMajorityQuiescingElectsLeader.1756994241738102-10634-0/minicluster-data/master-0-root
uuid: "2916f5d8da364a29ad8c7bfb96e06177"
format_stamp: "Formatted at 2025-09-04 13:57:31 on dist-test-slave-jx76"
I20250904 13:57:31.084775 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestMajorityQuiescingElectsLeader.1756994241738102-10634-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestMajorityQuiescingElectsLeader.1756994241738102-10634-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestMajorityQuiescingElectsLeader.1756994241738102-10634-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:31.097689 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:31.098837 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:31.135401 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.190:45057
I20250904 13:57:31.135489 11507 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.190:45057 every 8 connection(s)
I20250904 13:57:31.139249 11508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:31.149557 11508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177: Bootstrap starting.
I20250904 13:57:31.153899 11508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:31.158092 11508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177: No bootstrap required, opened a new log
I20250904 13:57:31.160045 11508 raft_consensus.cc:357] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2916f5d8da364a29ad8c7bfb96e06177" member_type: VOTER }
I20250904 13:57:31.160431 11508 raft_consensus.cc:383] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:31.160609 11508 raft_consensus.cc:738] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2916f5d8da364a29ad8c7bfb96e06177, State: Initialized, Role: FOLLOWER
I20250904 13:57:31.161075 11508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2916f5d8da364a29ad8c7bfb96e06177" member_type: VOTER }
I20250904 13:57:31.161480 11508 raft_consensus.cc:397] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:57:31.161670 11508 raft_consensus.cc:491] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250904 13:57:31.161893 11508 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:31.166356 11508 raft_consensus.cc:513] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2916f5d8da364a29ad8c7bfb96e06177" member_type: VOTER }
I20250904 13:57:31.166884 11508 leader_election.cc:304] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2916f5d8da364a29ad8c7bfb96e06177; no voters:
I20250904 13:57:31.167979 11508 leader_election.cc:290] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:57:31.168274 11511 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:31.169674 11511 raft_consensus.cc:695] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177 [term 1 LEADER]: Becoming Leader. State: Replica: 2916f5d8da364a29ad8c7bfb96e06177, State: Running, Role: LEADER
I20250904 13:57:31.170432 11511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2916f5d8da364a29ad8c7bfb96e06177" member_type: VOTER }
I20250904 13:57:31.171063 11508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 13:57:31.173605 11513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2916f5d8da364a29ad8c7bfb96e06177. Latest consensus state: current_term: 1 leader_uuid: "2916f5d8da364a29ad8c7bfb96e06177" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2916f5d8da364a29ad8c7bfb96e06177" member_type: VOTER } }
I20250904 13:57:31.173528 11512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2916f5d8da364a29ad8c7bfb96e06177" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2916f5d8da364a29ad8c7bfb96e06177" member_type: VOTER } }
I20250904 13:57:31.174266 11513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177 [sys.catalog]: This master's current role is: LEADER
I20250904 13:57:31.174625 11512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177 [sys.catalog]: This master's current role is: LEADER
I20250904 13:57:31.183440 11516 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 13:57:31.189302 11516 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 13:57:31.190500 10634 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 13:57:31.198536 11516 catalog_manager.cc:1349] Generated new cluster ID: edde6c801ea74233b33d4f37c55ff998
I20250904 13:57:31.198860 11516 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 13:57:31.215801 11516 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 13:57:31.217136 11516 catalog_manager.cc:1506] Loading token signing keys...
I20250904 13:57:31.229848 11516 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177: Generated new TSK 0
I20250904 13:57:31.230573 11516 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 13:57:31.257225 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:31.263720 11529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:31.265081 11530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:31.267037 10634 server_base.cc:1047] running on GCE node
W20250904 13:57:31.267330 11532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:31.268271 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:31.268468 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:31.268595 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994251268584 us; error 0 us; skew 500 ppm
I20250904 13:57:31.269029 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:31.271375 10634 webserver.cc:480] Webserver started at http://127.10.98.129:38163/ using document root <none> and password file <none>
I20250904 13:57:31.271862 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:31.272019 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:31.272243 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:31.273381 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestMajorityQuiescingElectsLeader.1756994241738102-10634-0/minicluster-data/ts-0-root/instance:
uuid: "78df136d55bd47d39893bf83e7c72285"
format_stamp: "Formatted at 2025-09-04 13:57:31 on dist-test-slave-jx76"
I20250904 13:57:31.278095 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20250904 13:57:31.281715 11537 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:31.282624 10634 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20250904 13:57:31.282886 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestMajorityQuiescingElectsLeader.1756994241738102-10634-0/minicluster-data/ts-0-root
uuid: "78df136d55bd47d39893bf83e7c72285"
format_stamp: "Formatted at 2025-09-04 13:57:31 on dist-test-slave-jx76"
I20250904 13:57:31.283140 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestMajorityQuiescingElectsLeader.1756994241738102-10634-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestMajorityQuiescingElectsLeader.1756994241738102-10634-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestMajorityQuiescingElectsLeader.1756994241738102-10634-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:31.299978 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:31.301153 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:31.302834 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:57:31.305500 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:57:31.305711 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:31.305950 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:57:31.306102 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:31.345172 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.129:33225
I20250904 13:57:31.345264 11599 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.129:33225 every 8 connection(s)
I20250904 13:57:31.349812 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:31.357023 11604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:31.359141 11605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:31.361497 11607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:31.362100 10634 server_base.cc:1047] running on GCE node
I20250904 13:57:31.363241 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:31.363440 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:31.363648 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994251363627 us; error 0 us; skew 500 ppm
I20250904 13:57:31.364285 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:31.366097 11600 heartbeater.cc:344] Connected to a master server at 127.10.98.190:45057
I20250904 13:57:31.366498 11600 heartbeater.cc:461] Registering TS with master...
I20250904 13:57:31.367111 10634 webserver.cc:480] Webserver started at http://127.10.98.130:45261/ using document root <none> and password file <none>
I20250904 13:57:31.367292 11600 heartbeater.cc:507] Master 127.10.98.190:45057 requested a full tablet report, sending...
I20250904 13:57:31.367663 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:31.367883 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:31.368155 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:31.369581 11473 ts_manager.cc:194] Registered new tserver with Master: 78df136d55bd47d39893bf83e7c72285 (127.10.98.129:33225)
I20250904 13:57:31.369596 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestMajorityQuiescingElectsLeader.1756994241738102-10634-0/minicluster-data/ts-1-root/instance:
uuid: "7705e1fcdcd448278ed3a0853221026b"
format_stamp: "Formatted at 2025-09-04 13:57:31 on dist-test-slave-jx76"
I20250904 13:57:31.372326 11473 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50998
I20250904 13:57:31.375766 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20250904 13:57:31.379298 11612 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:31.380209 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250904 13:57:31.380523 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestMajorityQuiescingElectsLeader.1756994241738102-10634-0/minicluster-data/ts-1-root
uuid: "7705e1fcdcd448278ed3a0853221026b"
format_stamp: "Formatted at 2025-09-04 13:57:31 on dist-test-slave-jx76"
I20250904 13:57:31.380801 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestMajorityQuiescingElectsLeader.1756994241738102-10634-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestMajorityQuiescingElectsLeader.1756994241738102-10634-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestMajorityQuiescingElectsLeader.1756994241738102-10634-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:31.407716 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:31.408839 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:31.410557 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:57:31.412952 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:57:31.413146 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:31.413331 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:57:31.413463 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:31.454453 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.130:36859
I20250904 13:57:31.454547 11674 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.130:36859 every 8 connection(s)
I20250904 13:57:31.459195 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:31.465942 11678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:31.467402 11679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:31.470443 11681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:31.471819 11675 heartbeater.cc:344] Connected to a master server at 127.10.98.190:45057
I20250904 13:57:31.472280 10634 server_base.cc:1047] running on GCE node
I20250904 13:57:31.472325 11675 heartbeater.cc:461] Registering TS with master...
I20250904 13:57:31.473248 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:31.473479 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:31.473400 11675 heartbeater.cc:507] Master 127.10.98.190:45057 requested a full tablet report, sending...
I20250904 13:57:31.473773 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994251473755 us; error 0 us; skew 500 ppm
I20250904 13:57:31.474378 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:31.476047 11473 ts_manager.cc:194] Registered new tserver with Master: 7705e1fcdcd448278ed3a0853221026b (127.10.98.130:36859)
I20250904 13:57:31.477138 10634 webserver.cc:480] Webserver started at http://127.10.98.131:37533/ using document root <none> and password file <none>
I20250904 13:57:31.477662 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:31.477614 11473 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51010
I20250904 13:57:31.477980 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:31.478291 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:31.479491 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestMajorityQuiescingElectsLeader.1756994241738102-10634-0/minicluster-data/ts-2-root/instance:
uuid: "4b003232b1094a6a9bc8f11817dcd4d9"
format_stamp: "Formatted at 2025-09-04 13:57:31 on dist-test-slave-jx76"
I20250904 13:57:31.484054 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250904 13:57:31.487481 11686 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:31.488220 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250904 13:57:31.488510 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestMajorityQuiescingElectsLeader.1756994241738102-10634-0/minicluster-data/ts-2-root
uuid: "4b003232b1094a6a9bc8f11817dcd4d9"
format_stamp: "Formatted at 2025-09-04 13:57:31 on dist-test-slave-jx76"
I20250904 13:57:31.488847 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestMajorityQuiescingElectsLeader.1756994241738102-10634-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestMajorityQuiescingElectsLeader.1756994241738102-10634-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestMajorityQuiescingElectsLeader.1756994241738102-10634-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:31.506242 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:31.507361 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:31.508816 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:57:31.511149 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:57:31.511338 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:31.511605 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:57:31.511755 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:31.550405 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.131:39025
I20250904 13:57:31.550519 11748 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.131:39025 every 8 connection(s)
I20250904 13:57:31.563925 11749 heartbeater.cc:344] Connected to a master server at 127.10.98.190:45057
I20250904 13:57:31.564309 11749 heartbeater.cc:461] Registering TS with master...
I20250904 13:57:31.565090 11749 heartbeater.cc:507] Master 127.10.98.190:45057 requested a full tablet report, sending...
I20250904 13:57:31.567111 11473 ts_manager.cc:194] Registered new tserver with Master: 4b003232b1094a6a9bc8f11817dcd4d9 (127.10.98.131:39025)
I20250904 13:57:31.567355 10634 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013694803s
I20250904 13:57:31.568543 11473 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51018
I20250904 13:57:31.600428 10634 test_util.cc:276] Using random seed: 440218132
I20250904 13:57:31.623965 11473 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51040:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 13:57:31.626012 11473 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 13:57:31.662837 11640 tablet_service.cc:1468] Processing CreateTablet for tablet 1727508a4a92433eb701cf6b1502a749 (DEFAULT_TABLE table=test-workload [id=441f12c0f37940bba79dfad63ec1e9e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:57:31.664227 11640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1727508a4a92433eb701cf6b1502a749. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:31.669103 11565 tablet_service.cc:1468] Processing CreateTablet for tablet 1727508a4a92433eb701cf6b1502a749 (DEFAULT_TABLE table=test-workload [id=441f12c0f37940bba79dfad63ec1e9e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:57:31.669799 11714 tablet_service.cc:1468] Processing CreateTablet for tablet 1727508a4a92433eb701cf6b1502a749 (DEFAULT_TABLE table=test-workload [id=441f12c0f37940bba79dfad63ec1e9e0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:57:31.670317 11565 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1727508a4a92433eb701cf6b1502a749. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:31.671118 11714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1727508a4a92433eb701cf6b1502a749. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:31.682399 11770 tablet_bootstrap.cc:492] T 1727508a4a92433eb701cf6b1502a749 P 7705e1fcdcd448278ed3a0853221026b: Bootstrap starting.
I20250904 13:57:31.685446 11771 tablet_bootstrap.cc:492] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285: Bootstrap starting.
I20250904 13:57:31.688694 11772 tablet_bootstrap.cc:492] T 1727508a4a92433eb701cf6b1502a749 P 4b003232b1094a6a9bc8f11817dcd4d9: Bootstrap starting.
I20250904 13:57:31.690143 11770 tablet_bootstrap.cc:654] T 1727508a4a92433eb701cf6b1502a749 P 7705e1fcdcd448278ed3a0853221026b: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:31.692595 11771 tablet_bootstrap.cc:654] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:31.694919 11772 tablet_bootstrap.cc:654] T 1727508a4a92433eb701cf6b1502a749 P 4b003232b1094a6a9bc8f11817dcd4d9: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:31.695783 11770 tablet_bootstrap.cc:492] T 1727508a4a92433eb701cf6b1502a749 P 7705e1fcdcd448278ed3a0853221026b: No bootstrap required, opened a new log
I20250904 13:57:31.696336 11770 ts_tablet_manager.cc:1397] T 1727508a4a92433eb701cf6b1502a749 P 7705e1fcdcd448278ed3a0853221026b: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20250904 13:57:31.697549 11771 tablet_bootstrap.cc:492] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285: No bootstrap required, opened a new log
I20250904 13:57:31.698031 11771 ts_tablet_manager.cc:1397] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20250904 13:57:31.699316 11770 raft_consensus.cc:357] T 1727508a4a92433eb701cf6b1502a749 P 7705e1fcdcd448278ed3a0853221026b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7705e1fcdcd448278ed3a0853221026b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 36859 } } peers { permanent_uuid: "4b003232b1094a6a9bc8f11817dcd4d9" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39025 } } peers { permanent_uuid: "78df136d55bd47d39893bf83e7c72285" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 33225 } }
I20250904 13:57:31.700069 11770 raft_consensus.cc:383] T 1727508a4a92433eb701cf6b1502a749 P 7705e1fcdcd448278ed3a0853221026b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:31.700376 11770 raft_consensus.cc:738] T 1727508a4a92433eb701cf6b1502a749 P 7705e1fcdcd448278ed3a0853221026b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7705e1fcdcd448278ed3a0853221026b, State: Initialized, Role: FOLLOWER
I20250904 13:57:31.701272 11772 tablet_bootstrap.cc:492] T 1727508a4a92433eb701cf6b1502a749 P 4b003232b1094a6a9bc8f11817dcd4d9: No bootstrap required, opened a new log
I20250904 13:57:31.701150 11770 consensus_queue.cc:260] T 1727508a4a92433eb701cf6b1502a749 P 7705e1fcdcd448278ed3a0853221026b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7705e1fcdcd448278ed3a0853221026b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 36859 } } peers { permanent_uuid: "4b003232b1094a6a9bc8f11817dcd4d9" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39025 } } peers { permanent_uuid: "78df136d55bd47d39893bf83e7c72285" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 33225 } }
I20250904 13:57:31.701272 11771 raft_consensus.cc:357] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7705e1fcdcd448278ed3a0853221026b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 36859 } } peers { permanent_uuid: "4b003232b1094a6a9bc8f11817dcd4d9" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39025 } } peers { permanent_uuid: "78df136d55bd47d39893bf83e7c72285" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 33225 } }
I20250904 13:57:31.701973 11772 ts_tablet_manager.cc:1397] T 1727508a4a92433eb701cf6b1502a749 P 4b003232b1094a6a9bc8f11817dcd4d9: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20250904 13:57:31.702235 11771 raft_consensus.cc:383] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:31.702564 11771 raft_consensus.cc:738] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78df136d55bd47d39893bf83e7c72285, State: Initialized, Role: FOLLOWER
I20250904 13:57:31.703943 11675 heartbeater.cc:499] Master 127.10.98.190:45057 was elected leader, sending a full tablet report...
I20250904 13:57:31.704463 11770 ts_tablet_manager.cc:1428] T 1727508a4a92433eb701cf6b1502a749 P 7705e1fcdcd448278ed3a0853221026b: Time spent starting tablet: real 0.008s user 0.007s sys 0.001s
I20250904 13:57:31.703480 11771 consensus_queue.cc:260] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7705e1fcdcd448278ed3a0853221026b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 36859 } } peers { permanent_uuid: "4b003232b1094a6a9bc8f11817dcd4d9" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39025 } } peers { permanent_uuid: "78df136d55bd47d39893bf83e7c72285" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 33225 } }
I20250904 13:57:31.704901 11772 raft_consensus.cc:357] T 1727508a4a92433eb701cf6b1502a749 P 4b003232b1094a6a9bc8f11817dcd4d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7705e1fcdcd448278ed3a0853221026b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 36859 } } peers { permanent_uuid: "4b003232b1094a6a9bc8f11817dcd4d9" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39025 } } peers { permanent_uuid: "78df136d55bd47d39893bf83e7c72285" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 33225 } }
I20250904 13:57:31.705664 11772 raft_consensus.cc:383] T 1727508a4a92433eb701cf6b1502a749 P 4b003232b1094a6a9bc8f11817dcd4d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:31.705987 11772 raft_consensus.cc:738] T 1727508a4a92433eb701cf6b1502a749 P 4b003232b1094a6a9bc8f11817dcd4d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b003232b1094a6a9bc8f11817dcd4d9, State: Initialized, Role: FOLLOWER
I20250904 13:57:31.706756 11772 consensus_queue.cc:260] T 1727508a4a92433eb701cf6b1502a749 P 4b003232b1094a6a9bc8f11817dcd4d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7705e1fcdcd448278ed3a0853221026b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 36859 } } peers { permanent_uuid: "4b003232b1094a6a9bc8f11817dcd4d9" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39025 } } peers { permanent_uuid: "78df136d55bd47d39893bf83e7c72285" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 33225 } }
I20250904 13:57:31.707274 11600 heartbeater.cc:499] Master 127.10.98.190:45057 was elected leader, sending a full tablet report...
I20250904 13:57:31.707958 11771 ts_tablet_manager.cc:1428] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285: Time spent starting tablet: real 0.009s user 0.005s sys 0.002s
I20250904 13:57:31.710896 11777 raft_consensus.cc:491] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:31.711166 11772 ts_tablet_manager.cc:1428] T 1727508a4a92433eb701cf6b1502a749 P 4b003232b1094a6a9bc8f11817dcd4d9: Time spent starting tablet: real 0.009s user 0.004s sys 0.003s
I20250904 13:57:31.711444 11777 raft_consensus.cc:513] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7705e1fcdcd448278ed3a0853221026b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 36859 } } peers { permanent_uuid: "4b003232b1094a6a9bc8f11817dcd4d9" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39025 } } peers { permanent_uuid: "78df136d55bd47d39893bf83e7c72285" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 33225 } }
I20250904 13:57:31.714089 11777 leader_election.cc:290] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7705e1fcdcd448278ed3a0853221026b (127.10.98.130:36859), 4b003232b1094a6a9bc8f11817dcd4d9 (127.10.98.131:39025)
I20250904 13:57:31.711932 11749 heartbeater.cc:499] Master 127.10.98.190:45057 was elected leader, sending a full tablet report...
I20250904 13:57:31.727360 11650 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1727508a4a92433eb701cf6b1502a749" candidate_uuid: "78df136d55bd47d39893bf83e7c72285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7705e1fcdcd448278ed3a0853221026b" is_pre_election: true
I20250904 13:57:31.727991 11650 raft_consensus.cc:2466] T 1727508a4a92433eb701cf6b1502a749 P 7705e1fcdcd448278ed3a0853221026b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78df136d55bd47d39893bf83e7c72285 in term 0.
I20250904 13:57:31.729113 11538 leader_election.cc:304] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7705e1fcdcd448278ed3a0853221026b, 78df136d55bd47d39893bf83e7c72285; no voters:
I20250904 13:57:31.729831 11777 raft_consensus.cc:2802] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:57:31.730247 11777 raft_consensus.cc:491] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:57:31.730593 11777 raft_consensus.cc:3058] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:31.732594 11724 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1727508a4a92433eb701cf6b1502a749" candidate_uuid: "78df136d55bd47d39893bf83e7c72285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b003232b1094a6a9bc8f11817dcd4d9" is_pre_election: true
I20250904 13:57:31.733143 11724 raft_consensus.cc:2466] T 1727508a4a92433eb701cf6b1502a749 P 4b003232b1094a6a9bc8f11817dcd4d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78df136d55bd47d39893bf83e7c72285 in term 0.
I20250904 13:57:31.737116 11777 raft_consensus.cc:513] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7705e1fcdcd448278ed3a0853221026b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 36859 } } peers { permanent_uuid: "4b003232b1094a6a9bc8f11817dcd4d9" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39025 } } peers { permanent_uuid: "78df136d55bd47d39893bf83e7c72285" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 33225 } }
I20250904 13:57:31.738641 11777 leader_election.cc:290] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 [CANDIDATE]: Term 1 election: Requested vote from peers 7705e1fcdcd448278ed3a0853221026b (127.10.98.130:36859), 4b003232b1094a6a9bc8f11817dcd4d9 (127.10.98.131:39025)
I20250904 13:57:31.739547 11650 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1727508a4a92433eb701cf6b1502a749" candidate_uuid: "78df136d55bd47d39893bf83e7c72285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7705e1fcdcd448278ed3a0853221026b"
I20250904 13:57:31.739578 11724 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1727508a4a92433eb701cf6b1502a749" candidate_uuid: "78df136d55bd47d39893bf83e7c72285" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b003232b1094a6a9bc8f11817dcd4d9"
I20250904 13:57:31.740120 11650 raft_consensus.cc:3058] T 1727508a4a92433eb701cf6b1502a749 P 7705e1fcdcd448278ed3a0853221026b [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:31.740159 11724 raft_consensus.cc:3058] T 1727508a4a92433eb701cf6b1502a749 P 4b003232b1094a6a9bc8f11817dcd4d9 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:31.744967 11650 raft_consensus.cc:2466] T 1727508a4a92433eb701cf6b1502a749 P 7705e1fcdcd448278ed3a0853221026b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78df136d55bd47d39893bf83e7c72285 in term 1.
I20250904 13:57:31.744969 11724 raft_consensus.cc:2466] T 1727508a4a92433eb701cf6b1502a749 P 4b003232b1094a6a9bc8f11817dcd4d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78df136d55bd47d39893bf83e7c72285 in term 1.
I20250904 13:57:31.746089 11539 leader_election.cc:304] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b003232b1094a6a9bc8f11817dcd4d9, 78df136d55bd47d39893bf83e7c72285; no voters:
I20250904 13:57:31.746815 11777 raft_consensus.cc:2802] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:31.747653 11777 raft_consensus.cc:695] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 [term 1 LEADER]: Becoming Leader. State: Replica: 78df136d55bd47d39893bf83e7c72285, State: Running, Role: LEADER
I20250904 13:57:31.748308 11777 consensus_queue.cc:237] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7705e1fcdcd448278ed3a0853221026b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 36859 } } peers { permanent_uuid: "4b003232b1094a6a9bc8f11817dcd4d9" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39025 } } peers { permanent_uuid: "78df136d55bd47d39893bf83e7c72285" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 33225 } }
I20250904 13:57:31.755415 11473 catalog_manager.cc:5582] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 reported cstate change: term changed from 0 to 1, leader changed from <none> to 78df136d55bd47d39893bf83e7c72285 (127.10.98.129). New cstate: current_term: 1 leader_uuid: "78df136d55bd47d39893bf83e7c72285" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7705e1fcdcd448278ed3a0853221026b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 36859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b003232b1094a6a9bc8f11817dcd4d9" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78df136d55bd47d39893bf83e7c72285" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 33225 } health_report { overall_health: HEALTHY } } }
I20250904 13:57:31.787341 10634 tablet_server.cc:178] TabletServer@127.10.98.129:0 shutting down...
I20250904 13:57:31.804529 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:57:31.805303 10634 tablet_replica.cc:331] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285: stopping tablet replica
I20250904 13:57:31.805796 10634 raft_consensus.cc:2241] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:57:31.806429 10634 pending_rounds.cc:70] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285: Trying to abort 1 pending ops.
I20250904 13:57:31.806622 10634 pending_rounds.cc:77] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7196648455167660032 op_type: NO_OP noop_request { }
I20250904 13:57:31.806907 10634 raft_consensus.cc:2887] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250904 13:57:31.807173 10634 raft_consensus.cc:2270] T 1727508a4a92433eb701cf6b1502a749 P 78df136d55bd47d39893bf83e7c72285 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:31.828104 10634 tablet_server.cc:195] TabletServer@127.10.98.129:0 shutdown complete.
I20250904 13:57:31.836756 10634 tablet_server.cc:178] TabletServer@127.10.98.130:0 shutting down...
I20250904 13:57:31.853554 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:57:31.854383 10634 tablet_replica.cc:331] T 1727508a4a92433eb701cf6b1502a749 P 7705e1fcdcd448278ed3a0853221026b: stopping tablet replica
I20250904 13:57:31.854992 10634 raft_consensus.cc:2241] T 1727508a4a92433eb701cf6b1502a749 P 7705e1fcdcd448278ed3a0853221026b [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:31.855384 10634 raft_consensus.cc:2270] T 1727508a4a92433eb701cf6b1502a749 P 7705e1fcdcd448278ed3a0853221026b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:31.874190 10634 tablet_server.cc:195] TabletServer@127.10.98.130:0 shutdown complete.
I20250904 13:57:31.882812 10634 tablet_server.cc:178] TabletServer@127.10.98.131:0 shutting down...
I20250904 13:57:31.899405 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:57:31.900270 10634 tablet_replica.cc:331] T 1727508a4a92433eb701cf6b1502a749 P 4b003232b1094a6a9bc8f11817dcd4d9: stopping tablet replica
I20250904 13:57:31.900821 10634 raft_consensus.cc:2241] T 1727508a4a92433eb701cf6b1502a749 P 4b003232b1094a6a9bc8f11817dcd4d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:31.901191 10634 raft_consensus.cc:2270] T 1727508a4a92433eb701cf6b1502a749 P 4b003232b1094a6a9bc8f11817dcd4d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:31.919745 10634 tablet_server.cc:195] TabletServer@127.10.98.131:0 shutdown complete.
I20250904 13:57:31.954870 10634 master.cc:561] Master@127.10.98.190:45057 shutting down...
I20250904 13:57:31.974452 10634 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:57:31.975389 10634 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:31.975821 10634 tablet_replica.cc:331] T 00000000000000000000000000000000 P 2916f5d8da364a29ad8c7bfb96e06177: stopping tablet replica
I20250904 13:57:31.997849 10634 master.cc:583] Master@127.10.98.190:45057 shutdown complete.
[ OK ] TServerQuiescingITest.TestMajorityQuiescingElectsLeader (974 ms)
[ RUN ] TServerQuiescingITest.TestDoesntAllowNewScans
I20250904 13:57:32.034344 10634 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.98.190:46623
I20250904 13:57:32.035570 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:32.041018 11783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:32.042050 11784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:32.042837 11786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:32.043121 10634 server_base.cc:1047] running on GCE node
I20250904 13:57:32.043933 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:32.044119 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:32.044243 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994252044226 us; error 0 us; skew 500 ppm
I20250904 13:57:32.044718 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:32.047132 10634 webserver.cc:480] Webserver started at http://127.10.98.190:37635/ using document root <none> and password file <none>
I20250904 13:57:32.047590 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:32.047750 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:32.047959 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:32.049223 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScans.1756994241738102-10634-0/minicluster-data/master-0-root/instance:
uuid: "339d985bf24841d4b06bfa10e97efef3"
format_stamp: "Formatted at 2025-09-04 13:57:32 on dist-test-slave-jx76"
I20250904 13:57:32.053978 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20250904 13:57:32.057457 11791 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:32.058267 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250904 13:57:32.058531 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScans.1756994241738102-10634-0/minicluster-data/master-0-root
uuid: "339d985bf24841d4b06bfa10e97efef3"
format_stamp: "Formatted at 2025-09-04 13:57:32 on dist-test-slave-jx76"
I20250904 13:57:32.058786 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScans.1756994241738102-10634-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScans.1756994241738102-10634-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScans.1756994241738102-10634-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:32.073837 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:32.075232 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:32.115708 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.190:46623
I20250904 13:57:32.115801 11842 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.190:46623 every 8 connection(s)
I20250904 13:57:32.119971 11843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:32.131120 11843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3: Bootstrap starting.
I20250904 13:57:32.136047 11843 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:32.140499 11843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3: No bootstrap required, opened a new log
I20250904 13:57:32.142802 11843 raft_consensus.cc:357] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "339d985bf24841d4b06bfa10e97efef3" member_type: VOTER }
I20250904 13:57:32.143390 11843 raft_consensus.cc:383] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:32.143596 11843 raft_consensus.cc:738] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 339d985bf24841d4b06bfa10e97efef3, State: Initialized, Role: FOLLOWER
I20250904 13:57:32.144106 11843 consensus_queue.cc:260] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "339d985bf24841d4b06bfa10e97efef3" member_type: VOTER }
I20250904 13:57:32.144552 11843 raft_consensus.cc:397] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:57:32.144744 11843 raft_consensus.cc:491] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250904 13:57:32.144986 11843 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:32.149459 11843 raft_consensus.cc:513] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "339d985bf24841d4b06bfa10e97efef3" member_type: VOTER }
I20250904 13:57:32.149983 11843 leader_election.cc:304] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 339d985bf24841d4b06bfa10e97efef3; no voters:
I20250904 13:57:32.151206 11843 leader_election.cc:290] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:57:32.151693 11846 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:32.153054 11846 raft_consensus.cc:695] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3 [term 1 LEADER]: Becoming Leader. State: Replica: 339d985bf24841d4b06bfa10e97efef3, State: Running, Role: LEADER
I20250904 13:57:32.153811 11846 consensus_queue.cc:237] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "339d985bf24841d4b06bfa10e97efef3" member_type: VOTER }
I20250904 13:57:32.154433 11843 sys_catalog.cc:564] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 13:57:32.157037 11848 sys_catalog.cc:455] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 339d985bf24841d4b06bfa10e97efef3. Latest consensus state: current_term: 1 leader_uuid: "339d985bf24841d4b06bfa10e97efef3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "339d985bf24841d4b06bfa10e97efef3" member_type: VOTER } }
I20250904 13:57:32.157020 11847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "339d985bf24841d4b06bfa10e97efef3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "339d985bf24841d4b06bfa10e97efef3" member_type: VOTER } }
I20250904 13:57:32.157760 11848 sys_catalog.cc:458] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3 [sys.catalog]: This master's current role is: LEADER
I20250904 13:57:32.157963 11847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3 [sys.catalog]: This master's current role is: LEADER
I20250904 13:57:32.162719 11851 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 13:57:32.167579 11851 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 13:57:32.170948 10634 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 13:57:32.176959 11851 catalog_manager.cc:1349] Generated new cluster ID: 4e5eb3cb27294d3088f0d6e0471989e5
I20250904 13:57:32.177277 11851 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 13:57:32.190729 11851 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 13:57:32.192099 11851 catalog_manager.cc:1506] Loading token signing keys...
I20250904 13:57:32.205798 11851 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3: Generated new TSK 0
I20250904 13:57:32.206465 11851 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 13:57:32.238396 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:32.245064 11864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:32.246589 11865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:32.248097 11867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:32.248512 10634 server_base.cc:1047] running on GCE node
I20250904 13:57:32.249356 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:32.249549 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:32.249673 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994252249661 us; error 0 us; skew 500 ppm
I20250904 13:57:32.250171 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:32.252661 10634 webserver.cc:480] Webserver started at http://127.10.98.129:43517/ using document root <none> and password file <none>
I20250904 13:57:32.253142 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:32.253310 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:32.253547 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:32.254822 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScans.1756994241738102-10634-0/minicluster-data/ts-0-root/instance:
uuid: "879e0dfb15664b77a6b083999416bdcb"
format_stamp: "Formatted at 2025-09-04 13:57:32 on dist-test-slave-jx76"
I20250904 13:57:32.259658 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250904 13:57:32.263362 11872 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:32.264266 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20250904 13:57:32.264581 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScans.1756994241738102-10634-0/minicluster-data/ts-0-root
uuid: "879e0dfb15664b77a6b083999416bdcb"
format_stamp: "Formatted at 2025-09-04 13:57:32 on dist-test-slave-jx76"
I20250904 13:57:32.264855 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScans.1756994241738102-10634-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScans.1756994241738102-10634-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScans.1756994241738102-10634-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:32.280877 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:32.282241 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:32.283820 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:57:32.286307 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:57:32.286536 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:32.286782 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:57:32.286952 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:32.329375 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.129:42253
I20250904 13:57:32.329478 11934 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.129:42253 every 8 connection(s)
I20250904 13:57:32.334363 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:32.342232 11939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:32.343098 11940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:32.346460 11942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:32.347375 10634 server_base.cc:1047] running on GCE node
I20250904 13:57:32.348495 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:32.348718 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:32.348894 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994252348874 us; error 0 us; skew 500 ppm
I20250904 13:57:32.348953 11935 heartbeater.cc:344] Connected to a master server at 127.10.98.190:46623
I20250904 13:57:32.349390 11935 heartbeater.cc:461] Registering TS with master...
I20250904 13:57:32.349720 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:32.350427 11935 heartbeater.cc:507] Master 127.10.98.190:46623 requested a full tablet report, sending...
I20250904 13:57:32.352681 10634 webserver.cc:480] Webserver started at http://127.10.98.130:35859/ using document root <none> and password file <none>
I20250904 13:57:32.352989 11808 ts_manager.cc:194] Registered new tserver with Master: 879e0dfb15664b77a6b083999416bdcb (127.10.98.129:42253)
I20250904 13:57:32.353214 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:32.353478 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:32.353757 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:32.354871 11808 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59008
I20250904 13:57:32.355067 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScans.1756994241738102-10634-0/minicluster-data/ts-1-root/instance:
uuid: "3361681465e64fa38e0e26f5dfe06510"
format_stamp: "Formatted at 2025-09-04 13:57:32 on dist-test-slave-jx76"
I20250904 13:57:32.360715 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20250904 13:57:32.364413 11947 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:32.365249 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250904 13:57:32.365568 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScans.1756994241738102-10634-0/minicluster-data/ts-1-root
uuid: "3361681465e64fa38e0e26f5dfe06510"
format_stamp: "Formatted at 2025-09-04 13:57:32 on dist-test-slave-jx76"
I20250904 13:57:32.365880 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScans.1756994241738102-10634-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScans.1756994241738102-10634-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScans.1756994241738102-10634-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:32.381544 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:32.382984 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:32.384490 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:57:32.386900 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:57:32.387113 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:32.387346 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:57:32.387501 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:32.430433 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.130:46863
I20250904 13:57:32.430529 12009 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.130:46863 every 8 connection(s)
I20250904 13:57:32.435577 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:32.443117 12013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:32.444326 12014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:32.447553 10634 server_base.cc:1047] running on GCE node
W20250904 13:57:32.447810 12016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:32.448589 12010 heartbeater.cc:344] Connected to a master server at 127.10.98.190:46623
I20250904 13:57:32.449030 12010 heartbeater.cc:461] Registering TS with master...
I20250904 13:57:32.448990 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:32.449440 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:32.449657 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994252449639 us; error 0 us; skew 500 ppm
I20250904 13:57:32.450191 12010 heartbeater.cc:507] Master 127.10.98.190:46623 requested a full tablet report, sending...
I20250904 13:57:32.450484 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:32.452885 11808 ts_manager.cc:194] Registered new tserver with Master: 3361681465e64fa38e0e26f5dfe06510 (127.10.98.130:46863)
I20250904 13:57:32.454648 10634 webserver.cc:480] Webserver started at http://127.10.98.131:41733/ using document root <none> and password file <none>
I20250904 13:57:32.454983 11808 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59014
I20250904 13:57:32.455286 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:32.455551 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:32.455874 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:32.457693 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScans.1756994241738102-10634-0/minicluster-data/ts-2-root/instance:
uuid: "43d8863bc4f84e4390efb132ff984eea"
format_stamp: "Formatted at 2025-09-04 13:57:32 on dist-test-slave-jx76"
I20250904 13:57:32.462803 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20250904 13:57:32.466650 12021 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:32.467492 10634 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20250904 13:57:32.467828 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScans.1756994241738102-10634-0/minicluster-data/ts-2-root
uuid: "43d8863bc4f84e4390efb132ff984eea"
format_stamp: "Formatted at 2025-09-04 13:57:32 on dist-test-slave-jx76"
I20250904 13:57:32.468143 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScans.1756994241738102-10634-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScans.1756994241738102-10634-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScans.1756994241738102-10634-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:32.482090 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:32.483420 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:32.484995 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:57:32.487447 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:57:32.487672 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:32.487908 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:57:32.488070 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:32.529562 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.131:33643
I20250904 13:57:32.529664 12083 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.131:33643 every 8 connection(s)
I20250904 13:57:32.543596 12084 heartbeater.cc:344] Connected to a master server at 127.10.98.190:46623
I20250904 13:57:32.544092 12084 heartbeater.cc:461] Registering TS with master...
I20250904 13:57:32.544876 12084 heartbeater.cc:507] Master 127.10.98.190:46623 requested a full tablet report, sending...
I20250904 13:57:32.546907 11808 ts_manager.cc:194] Registered new tserver with Master: 43d8863bc4f84e4390efb132ff984eea (127.10.98.131:33643)
I20250904 13:57:32.547432 10634 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01466534s
I20250904 13:57:32.548285 11808 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59030
I20250904 13:57:32.581125 10634 test_util.cc:276] Using random seed: 441198825
I20250904 13:57:32.604420 11808 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59078:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 13:57:32.607646 11808 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 13:57:32.657656 12049 tablet_service.cc:1468] Processing CreateTablet for tablet 3fac07932c864f5ab0a495e1675cd7d0 (DEFAULT_TABLE table=test-workload [id=014c08f056924c03b48f607d9866a574]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:57:32.659195 12049 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fac07932c864f5ab0a495e1675cd7d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:32.660425 11900 tablet_service.cc:1468] Processing CreateTablet for tablet 3fac07932c864f5ab0a495e1675cd7d0 (DEFAULT_TABLE table=test-workload [id=014c08f056924c03b48f607d9866a574]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:57:32.661854 11900 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fac07932c864f5ab0a495e1675cd7d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:32.665143 11975 tablet_service.cc:1468] Processing CreateTablet for tablet 3fac07932c864f5ab0a495e1675cd7d0 (DEFAULT_TABLE table=test-workload [id=014c08f056924c03b48f607d9866a574]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:57:32.666515 11975 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3fac07932c864f5ab0a495e1675cd7d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:32.687302 12105 tablet_bootstrap.cc:492] T 3fac07932c864f5ab0a495e1675cd7d0 P 43d8863bc4f84e4390efb132ff984eea: Bootstrap starting.
I20250904 13:57:32.687512 12107 tablet_bootstrap.cc:492] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510: Bootstrap starting.
I20250904 13:57:32.689730 12106 tablet_bootstrap.cc:492] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb: Bootstrap starting.
I20250904 13:57:32.694037 12105 tablet_bootstrap.cc:654] T 3fac07932c864f5ab0a495e1675cd7d0 P 43d8863bc4f84e4390efb132ff984eea: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:32.696305 12107 tablet_bootstrap.cc:654] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:32.697067 12106 tablet_bootstrap.cc:654] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:32.701759 12105 tablet_bootstrap.cc:492] T 3fac07932c864f5ab0a495e1675cd7d0 P 43d8863bc4f84e4390efb132ff984eea: No bootstrap required, opened a new log
I20250904 13:57:32.702334 12105 ts_tablet_manager.cc:1397] T 3fac07932c864f5ab0a495e1675cd7d0 P 43d8863bc4f84e4390efb132ff984eea: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.009s
I20250904 13:57:32.703358 12106 tablet_bootstrap.cc:492] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb: No bootstrap required, opened a new log
I20250904 13:57:32.703866 12106 ts_tablet_manager.cc:1397] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.005s
I20250904 13:57:32.704260 12107 tablet_bootstrap.cc:492] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510: No bootstrap required, opened a new log
I20250904 13:57:32.704663 12107 ts_tablet_manager.cc:1397] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.005s
I20250904 13:57:32.705246 12105 raft_consensus.cc:357] T 3fac07932c864f5ab0a495e1675cd7d0 P 43d8863bc4f84e4390efb132ff984eea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d8863bc4f84e4390efb132ff984eea" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 33643 } } peers { permanent_uuid: "879e0dfb15664b77a6b083999416bdcb" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42253 } } peers { permanent_uuid: "3361681465e64fa38e0e26f5dfe06510" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46863 } }
I20250904 13:57:32.706017 12105 raft_consensus.cc:383] T 3fac07932c864f5ab0a495e1675cd7d0 P 43d8863bc4f84e4390efb132ff984eea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:32.706338 12105 raft_consensus.cc:738] T 3fac07932c864f5ab0a495e1675cd7d0 P 43d8863bc4f84e4390efb132ff984eea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43d8863bc4f84e4390efb132ff984eea, State: Initialized, Role: FOLLOWER
I20250904 13:57:32.706635 12106 raft_consensus.cc:357] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d8863bc4f84e4390efb132ff984eea" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 33643 } } peers { permanent_uuid: "879e0dfb15664b77a6b083999416bdcb" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42253 } } peers { permanent_uuid: "3361681465e64fa38e0e26f5dfe06510" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46863 } }
I20250904 13:57:32.706982 12107 raft_consensus.cc:357] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d8863bc4f84e4390efb132ff984eea" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 33643 } } peers { permanent_uuid: "879e0dfb15664b77a6b083999416bdcb" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42253 } } peers { permanent_uuid: "3361681465e64fa38e0e26f5dfe06510" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46863 } }
I20250904 13:57:32.707355 12106 raft_consensus.cc:383] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:32.707065 12105 consensus_queue.cc:260] T 3fac07932c864f5ab0a495e1675cd7d0 P 43d8863bc4f84e4390efb132ff984eea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d8863bc4f84e4390efb132ff984eea" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 33643 } } peers { permanent_uuid: "879e0dfb15664b77a6b083999416bdcb" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42253 } } peers { permanent_uuid: "3361681465e64fa38e0e26f5dfe06510" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46863 } }
I20250904 13:57:32.707619 12107 raft_consensus.cc:383] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:32.707676 12106 raft_consensus.cc:738] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 879e0dfb15664b77a6b083999416bdcb, State: Initialized, Role: FOLLOWER
I20250904 13:57:32.707980 12107 raft_consensus.cc:738] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3361681465e64fa38e0e26f5dfe06510, State: Initialized, Role: FOLLOWER
I20250904 13:57:32.708580 12106 consensus_queue.cc:260] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d8863bc4f84e4390efb132ff984eea" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 33643 } } peers { permanent_uuid: "879e0dfb15664b77a6b083999416bdcb" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42253 } } peers { permanent_uuid: "3361681465e64fa38e0e26f5dfe06510" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46863 } }
I20250904 13:57:32.710605 12084 heartbeater.cc:499] Master 127.10.98.190:46623 was elected leader, sending a full tablet report...
I20250904 13:57:32.711140 12105 ts_tablet_manager.cc:1428] T 3fac07932c864f5ab0a495e1675cd7d0 P 43d8863bc4f84e4390efb132ff984eea: Time spent starting tablet: real 0.008s user 0.007s sys 0.001s
I20250904 13:57:32.708941 12107 consensus_queue.cc:260] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d8863bc4f84e4390efb132ff984eea" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 33643 } } peers { permanent_uuid: "879e0dfb15664b77a6b083999416bdcb" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42253 } } peers { permanent_uuid: "3361681465e64fa38e0e26f5dfe06510" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46863 } }
I20250904 13:57:32.713696 11935 heartbeater.cc:499] Master 127.10.98.190:46623 was elected leader, sending a full tablet report...
I20250904 13:57:32.714366 12106 ts_tablet_manager.cc:1428] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb: Time spent starting tablet: real 0.010s user 0.005s sys 0.007s
I20250904 13:57:32.716158 12010 heartbeater.cc:499] Master 127.10.98.190:46623 was elected leader, sending a full tablet report...
I20250904 13:57:32.716912 12107 ts_tablet_manager.cc:1428] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510: Time spent starting tablet: real 0.012s user 0.008s sys 0.001s
I20250904 13:57:32.945232 12112 raft_consensus.cc:491] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:32.945767 12112 raft_consensus.cc:513] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d8863bc4f84e4390efb132ff984eea" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 33643 } } peers { permanent_uuid: "879e0dfb15664b77a6b083999416bdcb" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42253 } } peers { permanent_uuid: "3361681465e64fa38e0e26f5dfe06510" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46863 } }
I20250904 13:57:32.948086 12112 leader_election.cc:290] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43d8863bc4f84e4390efb132ff984eea (127.10.98.131:33643), 3361681465e64fa38e0e26f5dfe06510 (127.10.98.130:46863)
I20250904 13:57:32.951726 12113 raft_consensus.cc:491] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:32.952358 12113 raft_consensus.cc:513] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d8863bc4f84e4390efb132ff984eea" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 33643 } } peers { permanent_uuid: "879e0dfb15664b77a6b083999416bdcb" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42253 } } peers { permanent_uuid: "3361681465e64fa38e0e26f5dfe06510" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46863 } }
I20250904 13:57:32.955009 12113 leader_election.cc:290] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 43d8863bc4f84e4390efb132ff984eea (127.10.98.131:33643), 879e0dfb15664b77a6b083999416bdcb (127.10.98.129:42253)
I20250904 13:57:32.962177 12059 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3fac07932c864f5ab0a495e1675cd7d0" candidate_uuid: "879e0dfb15664b77a6b083999416bdcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43d8863bc4f84e4390efb132ff984eea" is_pre_election: true
I20250904 13:57:32.962915 12059 raft_consensus.cc:2466] T 3fac07932c864f5ab0a495e1675cd7d0 P 43d8863bc4f84e4390efb132ff984eea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 879e0dfb15664b77a6b083999416bdcb in term 0.
I20250904 13:57:32.964205 11876 leader_election.cc:304] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43d8863bc4f84e4390efb132ff984eea, 879e0dfb15664b77a6b083999416bdcb; no voters:
I20250904 13:57:32.965284 12112 raft_consensus.cc:2802] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:57:32.965723 12112 raft_consensus.cc:491] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:57:32.966068 12112 raft_consensus.cc:3058] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:32.966531 11985 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3fac07932c864f5ab0a495e1675cd7d0" candidate_uuid: "879e0dfb15664b77a6b083999416bdcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3361681465e64fa38e0e26f5dfe06510" is_pre_election: true
I20250904 13:57:32.967288 11985 raft_consensus.cc:2466] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 879e0dfb15664b77a6b083999416bdcb in term 0.
I20250904 13:57:32.972942 12059 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3fac07932c864f5ab0a495e1675cd7d0" candidate_uuid: "3361681465e64fa38e0e26f5dfe06510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43d8863bc4f84e4390efb132ff984eea" is_pre_election: true
I20250904 13:57:32.973519 12059 raft_consensus.cc:2466] T 3fac07932c864f5ab0a495e1675cd7d0 P 43d8863bc4f84e4390efb132ff984eea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3361681465e64fa38e0e26f5dfe06510 in term 0.
I20250904 13:57:32.974498 11951 leader_election.cc:304] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3361681465e64fa38e0e26f5dfe06510, 43d8863bc4f84e4390efb132ff984eea; no voters:
I20250904 13:57:32.975261 12113 raft_consensus.cc:2802] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:57:32.975612 12113 raft_consensus.cc:491] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:57:32.975946 12113 raft_consensus.cc:3058] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:32.976020 12112 raft_consensus.cc:513] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d8863bc4f84e4390efb132ff984eea" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 33643 } } peers { permanent_uuid: "879e0dfb15664b77a6b083999416bdcb" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42253 } } peers { permanent_uuid: "3361681465e64fa38e0e26f5dfe06510" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46863 } }
I20250904 13:57:32.977329 11910 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3fac07932c864f5ab0a495e1675cd7d0" candidate_uuid: "3361681465e64fa38e0e26f5dfe06510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879e0dfb15664b77a6b083999416bdcb" is_pre_election: true
I20250904 13:57:32.978284 11910 raft_consensus.cc:2391] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 3361681465e64fa38e0e26f5dfe06510 in current term 1: Already voted for candidate 879e0dfb15664b77a6b083999416bdcb in this term.
I20250904 13:57:32.978200 12112 leader_election.cc:290] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [CANDIDATE]: Term 1 election: Requested vote from peers 43d8863bc4f84e4390efb132ff984eea (127.10.98.131:33643), 3361681465e64fa38e0e26f5dfe06510 (127.10.98.130:46863)
I20250904 13:57:32.979339 12059 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3fac07932c864f5ab0a495e1675cd7d0" candidate_uuid: "879e0dfb15664b77a6b083999416bdcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43d8863bc4f84e4390efb132ff984eea"
I20250904 13:57:32.979653 11985 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3fac07932c864f5ab0a495e1675cd7d0" candidate_uuid: "879e0dfb15664b77a6b083999416bdcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3361681465e64fa38e0e26f5dfe06510"
I20250904 13:57:32.979938 12059 raft_consensus.cc:3058] T 3fac07932c864f5ab0a495e1675cd7d0 P 43d8863bc4f84e4390efb132ff984eea [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:32.983076 12113 raft_consensus.cc:513] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d8863bc4f84e4390efb132ff984eea" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 33643 } } peers { permanent_uuid: "879e0dfb15664b77a6b083999416bdcb" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42253 } } peers { permanent_uuid: "3361681465e64fa38e0e26f5dfe06510" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46863 } }
I20250904 13:57:32.984278 11985 raft_consensus.cc:2391] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 879e0dfb15664b77a6b083999416bdcb in current term 1: Already voted for candidate 3361681465e64fa38e0e26f5dfe06510 in this term.
I20250904 13:57:32.985246 12113 leader_election.cc:290] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510 [CANDIDATE]: Term 1 election: Requested vote from peers 43d8863bc4f84e4390efb132ff984eea (127.10.98.131:33643), 879e0dfb15664b77a6b083999416bdcb (127.10.98.129:42253)
I20250904 13:57:32.986757 11910 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3fac07932c864f5ab0a495e1675cd7d0" candidate_uuid: "3361681465e64fa38e0e26f5dfe06510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "879e0dfb15664b77a6b083999416bdcb"
I20250904 13:57:32.986634 12058 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "3fac07932c864f5ab0a495e1675cd7d0" candidate_uuid: "3361681465e64fa38e0e26f5dfe06510" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "43d8863bc4f84e4390efb132ff984eea"
I20250904 13:57:32.987398 11910 raft_consensus.cc:2391] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 3361681465e64fa38e0e26f5dfe06510 in current term 1: Already voted for candidate 879e0dfb15664b77a6b083999416bdcb in this term.
I20250904 13:57:32.987298 12059 raft_consensus.cc:2466] T 3fac07932c864f5ab0a495e1675cd7d0 P 43d8863bc4f84e4390efb132ff984eea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 879e0dfb15664b77a6b083999416bdcb in term 1.
I20250904 13:57:32.988401 11951 leader_election.cc:304] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3361681465e64fa38e0e26f5dfe06510; no voters: 43d8863bc4f84e4390efb132ff984eea, 879e0dfb15664b77a6b083999416bdcb
I20250904 13:57:32.988399 11876 leader_election.cc:304] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 43d8863bc4f84e4390efb132ff984eea, 879e0dfb15664b77a6b083999416bdcb; no voters: 3361681465e64fa38e0e26f5dfe06510
I20250904 13:57:32.989384 12112 raft_consensus.cc:2802] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:32.990075 12113 raft_consensus.cc:2747] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250904 13:57:32.990607 12112 raft_consensus.cc:695] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [term 1 LEADER]: Becoming Leader. State: Replica: 879e0dfb15664b77a6b083999416bdcb, State: Running, Role: LEADER
I20250904 13:57:32.991282 12112 consensus_queue.cc:237] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d8863bc4f84e4390efb132ff984eea" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 33643 } } peers { permanent_uuid: "879e0dfb15664b77a6b083999416bdcb" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42253 } } peers { permanent_uuid: "3361681465e64fa38e0e26f5dfe06510" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46863 } }
I20250904 13:57:32.998067 11805 catalog_manager.cc:5582] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb reported cstate change: term changed from 0 to 1, leader changed from <none> to 879e0dfb15664b77a6b083999416bdcb (127.10.98.129). New cstate: current_term: 1 leader_uuid: "879e0dfb15664b77a6b083999416bdcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "43d8863bc4f84e4390efb132ff984eea" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 33643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "879e0dfb15664b77a6b083999416bdcb" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 42253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3361681465e64fa38e0e26f5dfe06510" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46863 } health_report { overall_health: UNKNOWN } } }
I20250904 13:57:33.149273 11985 raft_consensus.cc:1273] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510 [term 1 FOLLOWER]: Refusing update from remote peer 879e0dfb15664b77a6b083999416bdcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 13:57:33.149416 12059 raft_consensus.cc:1273] T 3fac07932c864f5ab0a495e1675cd7d0 P 43d8863bc4f84e4390efb132ff984eea [term 1 FOLLOWER]: Refusing update from remote peer 879e0dfb15664b77a6b083999416bdcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 13:57:33.151003 12112 consensus_queue.cc:1035] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3361681465e64fa38e0e26f5dfe06510" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:57:33.151880 12119 consensus_queue.cc:1035] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "43d8863bc4f84e4390efb132ff984eea" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 33643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:57:33.172044 12130 mvcc.cc:204] Tried to move back new op lower bound from 7196648460884414464 to 7196648460258988032. Current Snapshot: MvccSnapshot[applied={T|T < 7196648460884414464}]
I20250904 13:57:34.265362 10634 tablet_server.cc:178] TabletServer@127.10.98.131:0 shutting down...
W20250904 13:57:34.296125 11876 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 30 similar messages]
W20250904 13:57:34.341343 11876 consensus_peers.cc:489] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb -> Peer 43d8863bc4f84e4390efb132ff984eea (127.10.98.131:33643): Couldn't send request to peer 43d8863bc4f84e4390efb132ff984eea. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250904 13:57:34.354846 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:57:34.357118 10634 tablet_replica.cc:331] T 3fac07932c864f5ab0a495e1675cd7d0 P 43d8863bc4f84e4390efb132ff984eea: stopping tablet replica
I20250904 13:57:34.357909 10634 raft_consensus.cc:2241] T 3fac07932c864f5ab0a495e1675cd7d0 P 43d8863bc4f84e4390efb132ff984eea [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:34.358655 10634 raft_consensus.cc:2270] T 3fac07932c864f5ab0a495e1675cd7d0 P 43d8863bc4f84e4390efb132ff984eea [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:34.417615 10634 tablet_server.cc:195] TabletServer@127.10.98.131:0 shutdown complete.
I20250904 13:57:34.420091 12126 meta_cache.cc:1510] marking tablet server 43d8863bc4f84e4390efb132ff984eea (127.10.98.131:33643) as failed
W20250904 13:57:34.420703 12126 meta_cache.cc:302] tablet 3fac07932c864f5ab0a495e1675cd7d0: replica 43d8863bc4f84e4390efb132ff984eea (127.10.98.131:33643) has failed: Network error: TS failed: Client connection negotiation failed: client connection to 127.10.98.131:33643: connect: Connection refused (error 111)
I20250904 13:57:34.423894 12125 meta_cache.cc:1510] marking tablet server 43d8863bc4f84e4390efb132ff984eea (127.10.98.131:33643) as failed
I20250904 13:57:34.624519 10634 tablet_server.cc:178] TabletServer@127.10.98.129:0 shutting down...
I20250904 13:57:34.656378 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:57:34.657091 10634 tablet_replica.cc:331] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb: stopping tablet replica
I20250904 13:57:34.657793 10634 raft_consensus.cc:2241] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:57:34.659276 10634 raft_consensus.cc:2270] T 3fac07932c864f5ab0a495e1675cd7d0 P 879e0dfb15664b77a6b083999416bdcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:34.675277 10634 tablet_server.cc:195] TabletServer@127.10.98.129:0 shutdown complete.
I20250904 13:57:34.687999 10634 tablet_server.cc:178] TabletServer@127.10.98.130:0 shutting down...
I20250904 13:57:34.714478 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:57:34.715234 10634 tablet_replica.cc:331] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510: stopping tablet replica
I20250904 13:57:34.715812 10634 raft_consensus.cc:2241] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:34.716497 10634 raft_consensus.cc:2270] T 3fac07932c864f5ab0a495e1675cd7d0 P 3361681465e64fa38e0e26f5dfe06510 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:34.740206 10634 tablet_server.cc:195] TabletServer@127.10.98.130:0 shutdown complete.
I20250904 13:57:34.750089 10634 master.cc:561] Master@127.10.98.190:46623 shutting down...
I20250904 13:57:34.773114 10634 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:57:34.773689 10634 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:34.773990 10634 tablet_replica.cc:331] T 00000000000000000000000000000000 P 339d985bf24841d4b06bfa10e97efef3: stopping tablet replica
I20250904 13:57:34.792690 10634 master.cc:583] Master@127.10.98.190:46623 shutdown complete.
[ OK ] TServerQuiescingITest.TestDoesntAllowNewScans (2790 ms)
[ RUN ] TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly
I20250904 13:57:34.826220 10634 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.98.190:46161
I20250904 13:57:34.827818 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:34.833685 12149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:34.835058 12150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:34.837997 10634 server_base.cc:1047] running on GCE node
W20250904 13:57:34.838505 12152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:34.839388 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:34.839581 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:34.839733 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994254839715 us; error 0 us; skew 500 ppm
I20250904 13:57:34.840220 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:34.843147 10634 webserver.cc:480] Webserver started at http://127.10.98.190:34829/ using document root <none> and password file <none>
I20250904 13:57:34.843678 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:34.843878 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:34.844146 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:34.845315 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly.1756994241738102-10634-0/minicluster-data/master-0-root/instance:
uuid: "582ef7e148054118aa4140bd2bd04faa"
format_stamp: "Formatted at 2025-09-04 13:57:34 on dist-test-slave-jx76"
I20250904 13:57:34.850685 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20250904 13:57:34.854552 12157 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:34.855491 10634 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20250904 13:57:34.855794 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly.1756994241738102-10634-0/minicluster-data/master-0-root
uuid: "582ef7e148054118aa4140bd2bd04faa"
format_stamp: "Formatted at 2025-09-04 13:57:34 on dist-test-slave-jx76"
I20250904 13:57:34.856092 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly.1756994241738102-10634-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly.1756994241738102-10634-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly.1756994241738102-10634-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:34.876494 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:34.877626 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:34.924221 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.190:46161
I20250904 13:57:34.924400 12208 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.190:46161 every 8 connection(s)
I20250904 13:57:34.928664 12209 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:34.944608 12209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa: Bootstrap starting.
I20250904 13:57:34.950685 12209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:34.955987 12209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa: No bootstrap required, opened a new log
I20250904 13:57:34.958735 12209 raft_consensus.cc:357] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "582ef7e148054118aa4140bd2bd04faa" member_type: VOTER }
I20250904 13:57:34.959297 12209 raft_consensus.cc:383] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:34.959578 12209 raft_consensus.cc:738] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 582ef7e148054118aa4140bd2bd04faa, State: Initialized, Role: FOLLOWER
I20250904 13:57:34.960237 12209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "582ef7e148054118aa4140bd2bd04faa" member_type: VOTER }
I20250904 13:57:34.960837 12209 raft_consensus.cc:397] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:57:34.961119 12209 raft_consensus.cc:491] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250904 13:57:34.961431 12209 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:34.968483 12209 raft_consensus.cc:513] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "582ef7e148054118aa4140bd2bd04faa" member_type: VOTER }
I20250904 13:57:34.969228 12209 leader_election.cc:304] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 582ef7e148054118aa4140bd2bd04faa; no voters:
I20250904 13:57:34.970712 12209 leader_election.cc:290] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:57:34.971032 12212 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:34.973160 12212 raft_consensus.cc:695] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa [term 1 LEADER]: Becoming Leader. State: Replica: 582ef7e148054118aa4140bd2bd04faa, State: Running, Role: LEADER
I20250904 13:57:34.973933 12212 consensus_queue.cc:237] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "582ef7e148054118aa4140bd2bd04faa" member_type: VOTER }
I20250904 13:57:34.974362 12209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa [sys.catalog]: configured and running, proceeding with master startup.
I20250904 13:57:34.978698 12214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 582ef7e148054118aa4140bd2bd04faa. Latest consensus state: current_term: 1 leader_uuid: "582ef7e148054118aa4140bd2bd04faa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "582ef7e148054118aa4140bd2bd04faa" member_type: VOTER } }
I20250904 13:57:34.979302 12214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa [sys.catalog]: This master's current role is: LEADER
I20250904 13:57:34.981032 12213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "582ef7e148054118aa4140bd2bd04faa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "582ef7e148054118aa4140bd2bd04faa" member_type: VOTER } }
I20250904 13:57:34.981777 12213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa [sys.catalog]: This master's current role is: LEADER
I20250904 13:57:34.992717 12218 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 13:57:34.998533 12218 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 13:57:35.001143 10634 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 13:57:35.009339 12218 catalog_manager.cc:1349] Generated new cluster ID: 85a83c84770c49548af3ddafee48d547
I20250904 13:57:35.009670 12218 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 13:57:35.029428 12218 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 13:57:35.030879 12218 catalog_manager.cc:1506] Loading token signing keys...
I20250904 13:57:35.043479 12218 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa: Generated new TSK 0
I20250904 13:57:35.044190 12218 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 13:57:35.068743 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:35.075743 12230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:35.077312 12231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:35.080435 12233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:35.080698 10634 server_base.cc:1047] running on GCE node
I20250904 13:57:35.081678 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:35.081907 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:35.082103 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994255082086 us; error 0 us; skew 500 ppm
I20250904 13:57:35.082702 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:35.085776 10634 webserver.cc:480] Webserver started at http://127.10.98.129:41131/ using document root <none> and password file <none>
I20250904 13:57:35.086366 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:35.086582 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:35.086896 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:35.088096 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly.1756994241738102-10634-0/minicluster-data/ts-0-root/instance:
uuid: "464eb7dc213646849defa0776cfedb52"
format_stamp: "Formatted at 2025-09-04 13:57:35 on dist-test-slave-jx76"
I20250904 13:57:35.093096 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20250904 13:57:35.097074 12238 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:35.097958 10634 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20250904 13:57:35.098433 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly.1756994241738102-10634-0/minicluster-data/ts-0-root
uuid: "464eb7dc213646849defa0776cfedb52"
format_stamp: "Formatted at 2025-09-04 13:57:35 on dist-test-slave-jx76"
I20250904 13:57:35.098933 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly.1756994241738102-10634-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly.1756994241738102-10634-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly.1756994241738102-10634-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:35.116078 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:35.117719 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:35.119685 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:57:35.122972 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:57:35.123240 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:35.123529 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:57:35.123735 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:35.175137 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.129:36205
I20250904 13:57:35.175216 12300 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.129:36205 every 8 connection(s)
I20250904 13:57:35.183835 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:35.191879 12305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:35.192986 12306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:35.193650 12301 heartbeater.cc:344] Connected to a master server at 127.10.98.190:46161
I20250904 13:57:35.194181 12301 heartbeater.cc:461] Registering TS with master...
I20250904 13:57:35.195406 12301 heartbeater.cc:507] Master 127.10.98.190:46161 requested a full tablet report, sending...
I20250904 13:57:35.199229 12174 ts_manager.cc:194] Registered new tserver with Master: 464eb7dc213646849defa0776cfedb52 (127.10.98.129:36205)
I20250904 13:57:35.203142 12174 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38226
W20250904 13:57:35.210284 12308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:35.210326 10634 server_base.cc:1047] running on GCE node
I20250904 13:57:35.211591 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:35.211818 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:35.211972 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994255211957 us; error 0 us; skew 500 ppm
I20250904 13:57:35.212440 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:35.215361 10634 webserver.cc:480] Webserver started at http://127.10.98.130:45749/ using document root <none> and password file <none>
I20250904 13:57:35.215867 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:35.216053 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:35.216305 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:35.217393 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly.1756994241738102-10634-0/minicluster-data/ts-1-root/instance:
uuid: "724a4b4bc45048caa6e55b4db69ef398"
format_stamp: "Formatted at 2025-09-04 13:57:35 on dist-test-slave-jx76"
I20250904 13:57:35.222285 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20250904 13:57:35.226220 12313 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:35.227166 10634 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20250904 13:57:35.227469 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly.1756994241738102-10634-0/minicluster-data/ts-1-root
uuid: "724a4b4bc45048caa6e55b4db69ef398"
format_stamp: "Formatted at 2025-09-04 13:57:35 on dist-test-slave-jx76"
I20250904 13:57:35.227758 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly.1756994241738102-10634-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly.1756994241738102-10634-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly.1756994241738102-10634-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:35.245985 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:35.247152 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:35.248872 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:57:35.251361 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:57:35.251571 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:35.251819 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:57:35.251966 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:35.310232 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.130:46811
I20250904 13:57:35.310328 12375 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.130:46811 every 8 connection(s)
I20250904 13:57:35.315536 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:35.322542 12379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:35.325165 12380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:35.333468 12376 heartbeater.cc:344] Connected to a master server at 127.10.98.190:46161
I20250904 13:57:35.333734 10634 server_base.cc:1047] running on GCE node
I20250904 13:57:35.333997 12376 heartbeater.cc:461] Registering TS with master...
I20250904 13:57:35.335150 12376 heartbeater.cc:507] Master 127.10.98.190:46161 requested a full tablet report, sending...
W20250904 13:57:35.335541 12382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:35.336618 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:35.336843 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:35.337107 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994255337085 us; error 0 us; skew 500 ppm
I20250904 13:57:35.337622 12174 ts_manager.cc:194] Registered new tserver with Master: 724a4b4bc45048caa6e55b4db69ef398 (127.10.98.130:46811)
I20250904 13:57:35.337848 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:35.339768 12174 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38242
I20250904 13:57:35.343104 10634 webserver.cc:480] Webserver started at http://127.10.98.131:45531/ using document root <none> and password file <none>
I20250904 13:57:35.343647 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:35.343824 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:35.344094 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:35.345198 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly.1756994241738102-10634-0/minicluster-data/ts-2-root/instance:
uuid: "43cfdecc88a74213b3c25129a27e9a2c"
format_stamp: "Formatted at 2025-09-04 13:57:35 on dist-test-slave-jx76"
I20250904 13:57:35.350098 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20250904 13:57:35.354468 12387 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:35.355401 10634 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20250904 13:57:35.355712 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly.1756994241738102-10634-0/minicluster-data/ts-2-root
uuid: "43cfdecc88a74213b3c25129a27e9a2c"
format_stamp: "Formatted at 2025-09-04 13:57:35 on dist-test-slave-jx76"
I20250904 13:57:35.356017 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly.1756994241738102-10634-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly.1756994241738102-10634-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly.1756994241738102-10634-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:35.373505 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:35.374682 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:35.376384 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:57:35.379045 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:57:35.379258 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:35.379473 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:57:35.379624 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:35.437196 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.131:40633
I20250904 13:57:35.437309 12449 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.131:40633 every 8 connection(s)
I20250904 13:57:35.452248 12450 heartbeater.cc:344] Connected to a master server at 127.10.98.190:46161
I20250904 13:57:35.452652 12450 heartbeater.cc:461] Registering TS with master...
I20250904 13:57:35.453485 12450 heartbeater.cc:507] Master 127.10.98.190:46161 requested a full tablet report, sending...
I20250904 13:57:35.455597 12174 ts_manager.cc:194] Registered new tserver with Master: 43cfdecc88a74213b3c25129a27e9a2c (127.10.98.131:40633)
I20250904 13:57:35.456333 10634 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015389835s
I20250904 13:57:35.457492 12174 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38248
I20250904 13:57:35.499771 10634 test_util.cc:276] Using random seed: 444117466
I20250904 13:57:35.524643 12174 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38274:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 13:57:35.526527 12174 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 13:57:35.587029 12341 tablet_service.cc:1468] Processing CreateTablet for tablet eb9144f161da423f9fe899f96f54081c (DEFAULT_TABLE table=test-workload [id=37692ed6a9a847e9b4d5bdc8a5692ac9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:57:35.588445 12341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb9144f161da423f9fe899f96f54081c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:35.594154 12266 tablet_service.cc:1468] Processing CreateTablet for tablet eb9144f161da423f9fe899f96f54081c (DEFAULT_TABLE table=test-workload [id=37692ed6a9a847e9b4d5bdc8a5692ac9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:57:35.595543 12266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb9144f161da423f9fe899f96f54081c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:35.606977 12471 tablet_bootstrap.cc:492] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398: Bootstrap starting.
I20250904 13:57:35.607290 12415 tablet_service.cc:1468] Processing CreateTablet for tablet eb9144f161da423f9fe899f96f54081c (DEFAULT_TABLE table=test-workload [id=37692ed6a9a847e9b4d5bdc8a5692ac9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:57:35.608668 12415 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb9144f161da423f9fe899f96f54081c. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:35.616153 12471 tablet_bootstrap.cc:654] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:35.627373 12472 tablet_bootstrap.cc:492] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52: Bootstrap starting.
I20250904 13:57:35.629923 12474 tablet_bootstrap.cc:492] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c: Bootstrap starting.
I20250904 13:57:35.630402 12471 tablet_bootstrap.cc:492] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398: No bootstrap required, opened a new log
I20250904 13:57:35.630983 12471 ts_tablet_manager.cc:1397] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398: Time spent bootstrapping tablet: real 0.024s user 0.016s sys 0.000s
I20250904 13:57:35.633915 12471 raft_consensus.cc:357] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724a4b4bc45048caa6e55b4db69ef398" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46811 } } peers { permanent_uuid: "43cfdecc88a74213b3c25129a27e9a2c" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40633 } } peers { permanent_uuid: "464eb7dc213646849defa0776cfedb52" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36205 } }
I20250904 13:57:35.634894 12471 raft_consensus.cc:383] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:35.635274 12471 raft_consensus.cc:738] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 724a4b4bc45048caa6e55b4db69ef398, State: Initialized, Role: FOLLOWER
I20250904 13:57:35.635530 12472 tablet_bootstrap.cc:654] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:35.636117 12471 consensus_queue.cc:260] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724a4b4bc45048caa6e55b4db69ef398" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46811 } } peers { permanent_uuid: "43cfdecc88a74213b3c25129a27e9a2c" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40633 } } peers { permanent_uuid: "464eb7dc213646849defa0776cfedb52" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36205 } }
I20250904 13:57:35.637677 12474 tablet_bootstrap.cc:654] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:35.644456 12472 tablet_bootstrap.cc:492] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52: No bootstrap required, opened a new log
I20250904 13:57:35.645090 12472 ts_tablet_manager.cc:1397] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.010s
I20250904 13:57:35.647877 12472 raft_consensus.cc:357] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724a4b4bc45048caa6e55b4db69ef398" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46811 } } peers { permanent_uuid: "43cfdecc88a74213b3c25129a27e9a2c" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40633 } } peers { permanent_uuid: "464eb7dc213646849defa0776cfedb52" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36205 } }
I20250904 13:57:35.649817 12472 raft_consensus.cc:383] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:35.649981 12376 heartbeater.cc:499] Master 127.10.98.190:46161 was elected leader, sending a full tablet report...
I20250904 13:57:35.650262 12472 raft_consensus.cc:738] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 464eb7dc213646849defa0776cfedb52, State: Initialized, Role: FOLLOWER
I20250904 13:57:35.650959 12471 ts_tablet_manager.cc:1428] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398: Time spent starting tablet: real 0.020s user 0.007s sys 0.008s
I20250904 13:57:35.651371 12472 consensus_queue.cc:260] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724a4b4bc45048caa6e55b4db69ef398" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46811 } } peers { permanent_uuid: "43cfdecc88a74213b3c25129a27e9a2c" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40633 } } peers { permanent_uuid: "464eb7dc213646849defa0776cfedb52" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36205 } }
I20250904 13:57:35.656095 12474 tablet_bootstrap.cc:492] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c: No bootstrap required, opened a new log
I20250904 13:57:35.656800 12474 ts_tablet_manager.cc:1397] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c: Time spent bootstrapping tablet: real 0.028s user 0.012s sys 0.013s
I20250904 13:57:35.659921 12474 raft_consensus.cc:357] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724a4b4bc45048caa6e55b4db69ef398" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46811 } } peers { permanent_uuid: "43cfdecc88a74213b3c25129a27e9a2c" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40633 } } peers { permanent_uuid: "464eb7dc213646849defa0776cfedb52" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36205 } }
I20250904 13:57:35.660871 12474 raft_consensus.cc:383] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:35.661247 12474 raft_consensus.cc:738] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43cfdecc88a74213b3c25129a27e9a2c, State: Initialized, Role: FOLLOWER
I20250904 13:57:35.662272 12474 consensus_queue.cc:260] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724a4b4bc45048caa6e55b4db69ef398" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46811 } } peers { permanent_uuid: "43cfdecc88a74213b3c25129a27e9a2c" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40633 } } peers { permanent_uuid: "464eb7dc213646849defa0776cfedb52" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36205 } }
I20250904 13:57:35.668895 12301 heartbeater.cc:499] Master 127.10.98.190:46161 was elected leader, sending a full tablet report...
I20250904 13:57:35.669421 12472 ts_tablet_manager.cc:1428] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52: Time spent starting tablet: real 0.024s user 0.010s sys 0.009s
I20250904 13:57:35.671958 12450 heartbeater.cc:499] Master 127.10.98.190:46161 was elected leader, sending a full tablet report...
I20250904 13:57:35.672153 12474 ts_tablet_manager.cc:1428] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c: Time spent starting tablet: real 0.015s user 0.005s sys 0.008s
I20250904 13:57:35.682757 12479 raft_consensus.cc:491] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:35.683339 12479 raft_consensus.cc:513] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724a4b4bc45048caa6e55b4db69ef398" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46811 } } peers { permanent_uuid: "43cfdecc88a74213b3c25129a27e9a2c" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40633 } } peers { permanent_uuid: "464eb7dc213646849defa0776cfedb52" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36205 } }
I20250904 13:57:35.688274 12479 leader_election.cc:290] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 724a4b4bc45048caa6e55b4db69ef398 (127.10.98.130:46811), 464eb7dc213646849defa0776cfedb52 (127.10.98.129:36205)
I20250904 13:57:35.698349 12276 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "eb9144f161da423f9fe899f96f54081c" candidate_uuid: "43cfdecc88a74213b3c25129a27e9a2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464eb7dc213646849defa0776cfedb52" is_pre_election: true
I20250904 13:57:35.699237 12276 raft_consensus.cc:2466] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cfdecc88a74213b3c25129a27e9a2c in term 0.
I20250904 13:57:35.700398 12390 leader_election.cc:304] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cfdecc88a74213b3c25129a27e9a2c, 464eb7dc213646849defa0776cfedb52; no voters:
I20250904 13:57:35.701282 12479 raft_consensus.cc:2802] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:57:35.701440 12351 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "eb9144f161da423f9fe899f96f54081c" candidate_uuid: "43cfdecc88a74213b3c25129a27e9a2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "724a4b4bc45048caa6e55b4db69ef398" is_pre_election: true
I20250904 13:57:35.701726 12479 raft_consensus.cc:491] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:57:35.702250 12479 raft_consensus.cc:3058] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:35.702273 12351 raft_consensus.cc:2466] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 43cfdecc88a74213b3c25129a27e9a2c in term 0.
I20250904 13:57:35.708524 12479 raft_consensus.cc:513] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724a4b4bc45048caa6e55b4db69ef398" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46811 } } peers { permanent_uuid: "43cfdecc88a74213b3c25129a27e9a2c" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40633 } } peers { permanent_uuid: "464eb7dc213646849defa0776cfedb52" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36205 } }
I20250904 13:57:35.710199 12479 leader_election.cc:290] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [CANDIDATE]: Term 1 election: Requested vote from peers 724a4b4bc45048caa6e55b4db69ef398 (127.10.98.130:46811), 464eb7dc213646849defa0776cfedb52 (127.10.98.129:36205)
I20250904 13:57:35.711118 12351 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "eb9144f161da423f9fe899f96f54081c" candidate_uuid: "43cfdecc88a74213b3c25129a27e9a2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "724a4b4bc45048caa6e55b4db69ef398"
I20250904 13:57:35.711287 12276 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "eb9144f161da423f9fe899f96f54081c" candidate_uuid: "43cfdecc88a74213b3c25129a27e9a2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464eb7dc213646849defa0776cfedb52"
I20250904 13:57:35.711802 12276 raft_consensus.cc:3058] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:35.711802 12351 raft_consensus.cc:3058] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:35.719523 12276 raft_consensus.cc:2466] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cfdecc88a74213b3c25129a27e9a2c in term 1.
I20250904 13:57:35.719523 12351 raft_consensus.cc:2466] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 43cfdecc88a74213b3c25129a27e9a2c in term 1.
I20250904 13:57:35.720886 12391 leader_election.cc:304] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 43cfdecc88a74213b3c25129a27e9a2c, 724a4b4bc45048caa6e55b4db69ef398; no voters:
I20250904 13:57:35.721753 12479 raft_consensus.cc:2802] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:35.722990 12479 raft_consensus.cc:695] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [term 1 LEADER]: Becoming Leader. State: Replica: 43cfdecc88a74213b3c25129a27e9a2c, State: Running, Role: LEADER
I20250904 13:57:35.723917 12479 consensus_queue.cc:237] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724a4b4bc45048caa6e55b4db69ef398" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46811 } } peers { permanent_uuid: "43cfdecc88a74213b3c25129a27e9a2c" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40633 } } peers { permanent_uuid: "464eb7dc213646849defa0776cfedb52" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36205 } }
I20250904 13:57:35.733623 12173 catalog_manager.cc:5582] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c reported cstate change: term changed from 0 to 1, leader changed from <none> to 43cfdecc88a74213b3c25129a27e9a2c (127.10.98.131). New cstate: current_term: 1 leader_uuid: "43cfdecc88a74213b3c25129a27e9a2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724a4b4bc45048caa6e55b4db69ef398" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cfdecc88a74213b3c25129a27e9a2c" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40633 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "464eb7dc213646849defa0776cfedb52" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36205 } health_report { overall_health: UNKNOWN } } }
I20250904 13:57:35.889614 12351 raft_consensus.cc:1273] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 1 FOLLOWER]: Refusing update from remote peer 43cfdecc88a74213b3c25129a27e9a2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 13:57:35.889621 12276 raft_consensus.cc:1273] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 1 FOLLOWER]: Refusing update from remote peer 43cfdecc88a74213b3c25129a27e9a2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 13:57:35.892603 12479 consensus_queue.cc:1035] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "724a4b4bc45048caa6e55b4db69ef398" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:57:35.893759 12481 consensus_queue.cc:1035] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "464eb7dc213646849defa0776cfedb52" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250904 13:57:35.979362 12479 raft_consensus.cc:991] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c: : Instructing follower 724a4b4bc45048caa6e55b4db69ef398 to start an election
I20250904 13:57:35.980136 12481 raft_consensus.cc:1079] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [term 1 LEADER]: Signalling peer 724a4b4bc45048caa6e55b4db69ef398 to start an election
I20250904 13:57:35.985993 12350 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "eb9144f161da423f9fe899f96f54081c"
dest_uuid: "724a4b4bc45048caa6e55b4db69ef398"
from {username='slave'} at 127.0.0.1:55196
I20250904 13:57:36.006711 10634 tablet_server.cc:178] TabletServer@127.10.98.131:0 shutting down...
I20250904 13:57:36.027397 12479 raft_consensus.cc:991] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c: : Instructing follower 464eb7dc213646849defa0776cfedb52 to start an election
I20250904 13:57:36.029567 12498 raft_consensus.cc:1079] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [term 1 LEADER]: Signalling peer 464eb7dc213646849defa0776cfedb52 to start an election
I20250904 13:57:36.044674 12275 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "eb9144f161da423f9fe899f96f54081c"
dest_uuid: "464eb7dc213646849defa0776cfedb52"
from {username='slave'} at 127.0.0.1:37712
I20250904 13:57:36.061789 12350 raft_consensus.cc:491] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250904 13:57:36.062392 12350 raft_consensus.cc:3058] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 1 FOLLOWER]: Advancing to term 2
I20250904 13:57:36.069288 12479 raft_consensus.cc:991] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c: : Instructing follower 724a4b4bc45048caa6e55b4db69ef398 to start an election
I20250904 13:57:36.069804 12479 raft_consensus.cc:1079] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [term 1 LEADER]: Signalling peer 724a4b4bc45048caa6e55b4db69ef398 to start an election
I20250904 13:57:36.073904 12350 raft_consensus.cc:513] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724a4b4bc45048caa6e55b4db69ef398" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46811 } } peers { permanent_uuid: "43cfdecc88a74213b3c25129a27e9a2c" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40633 } } peers { permanent_uuid: "464eb7dc213646849defa0776cfedb52" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36205 } }
I20250904 13:57:36.075780 12351 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "eb9144f161da423f9fe899f96f54081c"
dest_uuid: "724a4b4bc45048caa6e55b4db69ef398"
from {username='slave'} at 127.0.0.1:55196
I20250904 13:57:36.077665 12351 raft_consensus.cc:491] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250904 13:57:36.081488 12351 raft_consensus.cc:3058] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 2 FOLLOWER]: Advancing to term 3
I20250904 13:57:36.089936 12350 leader_election.cc:290] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [CANDIDATE]: Term 2 election: Requested vote from peers 43cfdecc88a74213b3c25129a27e9a2c (127.10.98.131:40633), 464eb7dc213646849defa0776cfedb52 (127.10.98.129:36205)
I20250904 13:57:36.092716 12275 raft_consensus.cc:491] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250904 13:57:36.097926 12275 raft_consensus.cc:3058] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 1 FOLLOWER]: Advancing to term 2
I20250904 13:57:36.099017 12481 raft_consensus.cc:991] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c: : Instructing follower 464eb7dc213646849defa0776cfedb52 to start an election
I20250904 13:57:36.099467 12481 raft_consensus.cc:1079] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [term 1 LEADER]: Signalling peer 464eb7dc213646849defa0776cfedb52 to start an election
I20250904 13:57:36.101657 12276 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "eb9144f161da423f9fe899f96f54081c"
dest_uuid: "464eb7dc213646849defa0776cfedb52"
from {username='slave'} at 127.0.0.1:37712
I20250904 13:57:36.117177 12275 raft_consensus.cc:513] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724a4b4bc45048caa6e55b4db69ef398" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46811 } } peers { permanent_uuid: "43cfdecc88a74213b3c25129a27e9a2c" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40633 } } peers { permanent_uuid: "464eb7dc213646849defa0776cfedb52" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36205 } }
I20250904 13:57:36.125769 12276 raft_consensus.cc:491] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250904 13:57:36.127583 12276 raft_consensus.cc:3058] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 2 FOLLOWER]: Advancing to term 3
I20250904 13:57:36.126753 12351 raft_consensus.cc:513] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724a4b4bc45048caa6e55b4db69ef398" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46811 } } peers { permanent_uuid: "43cfdecc88a74213b3c25129a27e9a2c" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40633 } } peers { permanent_uuid: "464eb7dc213646849defa0776cfedb52" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36205 } }
I20250904 13:57:36.150507 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:57:36.146576 12351 leader_election.cc:290] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [CANDIDATE]: Term 3 election: Requested vote from peers 43cfdecc88a74213b3c25129a27e9a2c (127.10.98.131:40633), 464eb7dc213646849defa0776cfedb52 (127.10.98.129:36205)
I20250904 13:57:36.152779 10634 tablet_replica.cc:331] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c: stopping tablet replica
I20250904 13:57:36.153311 10634 raft_consensus.cc:2241] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:57:36.153877 10634 raft_consensus.cc:2270] T eb9144f161da423f9fe899f96f54081c P 43cfdecc88a74213b3c25129a27e9a2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:36.163255 12275 leader_election.cc:290] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [CANDIDATE]: Term 2 election: Requested vote from peers 724a4b4bc45048caa6e55b4db69ef398 (127.10.98.130:46811), 43cfdecc88a74213b3c25129a27e9a2c (127.10.98.131:40633)
I20250904 13:57:36.181020 12276 raft_consensus.cc:513] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724a4b4bc45048caa6e55b4db69ef398" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46811 } } peers { permanent_uuid: "43cfdecc88a74213b3c25129a27e9a2c" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40633 } } peers { permanent_uuid: "464eb7dc213646849defa0776cfedb52" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36205 } }
I20250904 13:57:36.196885 12276 leader_election.cc:290] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [CANDIDATE]: Term 3 election: Requested vote from peers 724a4b4bc45048caa6e55b4db69ef398 (127.10.98.130:46811), 43cfdecc88a74213b3c25129a27e9a2c (127.10.98.131:40633)
W20250904 13:57:36.214649 12317 leader_election.cc:336] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43cfdecc88a74213b3c25129a27e9a2c (127.10.98.131:40633): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20250904 13:57:36.218703 12317 leader_election.cc:336] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 43cfdecc88a74213b3c25129a27e9a2c (127.10.98.131:40633): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20250904 13:57:36.266726 12389 consensus_peers.cc:161] SignalRequest failed: Illegal state: peer closed
==================
WARNING: ThreadSanitizer: data race (pid=10634)
Write of size 8 at 0x7b4400038d48 by main thread:
#0 std::__1::__vector_base<kudu::MemTracker*, std::__1::allocator<kudu::MemTracker*> >::__destruct_at_end(kudu::MemTracker**) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/vector:429:12 (libkudu_util.so+0x3a8620)
#1 std::__1::__vector_base<kudu::MemTracker*, std::__1::allocator<kudu::MemTracker*> >::clear() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/vector:371:29 (libkudu_util.so+0x3a8534)
#2 std::__1::__vector_base<kudu::MemTracker*, std::__1::allocator<kudu::MemTracker*> >::~__vector_base() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/vector:465:9 (libkudu_util.so+0x3a832b)
#3 std::__1::vector<kudu::MemTracker*, std::__1::allocator<kudu::MemTracker*> >::~vector() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/vector:557:5 (libkudu_util.so+0x3a05e1)
#4 kudu::MemTracker::~MemTracker() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/mem_tracker.cc:83:1 (libkudu_util.so+0x3a0193)
#5 std::__1::__shared_ptr_emplace<std::__1::shared_ptr<kudu::MemTracker> enable_make_shared<kudu::MemTracker>::make_shared<long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<kudu::MemTracker>&>(long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<kudu::MemTracker>&)::make_shared_enabler, std::__1::allocator<std::__1::shared_ptr<kudu::MemTracker> enable_make_shared<kudu::MemTracker>::make_shared<long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<kudu::MemTracker>&>(long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<kudu::MemTracker>&)::make_shared_enabler> >::__on_zero_shared() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:3503:23 (libkudu_util.so+0x3a7455)
#6 std::__1::__shared_count::__release_shared() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:3341:9 (libtablet_test_util.so+0x35485)
#7 std::__1::__shared_weak_count::__release_shared() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:3383:27 (libtablet_test_util.so+0x35429)
#8 std::__1::shared_ptr<kudu::MemTracker>::~shared_ptr() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:4098:19 (libtablet_test_util.so+0x31c18)
#9 kudu::tablet::OpTracker::~OpTracker() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_tracker.cc:142:1 (libtablet.so+0x41c8f4)
#10 kudu::tablet::TabletReplica::~TabletReplica() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet_replica.cc:202:1 (libtablet.so+0x3cf6d5)
#11 kudu::tablet::TabletReplica::~TabletReplica() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet_replica.cc:197:33 (libtablet.so+0x3cf8e9)
#12 kudu::RefCountedThreadSafe<kudu::tablet::TabletReplica, kudu::DefaultRefCountedThreadSafeTraits<kudu::tablet::TabletReplica> >::DeleteInternal(kudu::tablet::TabletReplica const*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/gutil/ref_counted.h:153:44 (libitest_util.so+0x1002d7)
#13 kudu::DefaultRefCountedThreadSafeTraits<kudu::tablet::TabletReplica>::Destruct(kudu::tablet::TabletReplica const*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/gutil/ref_counted.h:116:5 (libitest_util.so+0x100299)
#14 kudu::RefCountedThreadSafe<kudu::tablet::TabletReplica, kudu::DefaultRefCountedThreadSafeTraits<kudu::tablet::TabletReplica> >::Release() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/gutil/ref_counted.h:144:7 (libitest_util.so+0x100259)
#15 scoped_refptr<kudu::tablet::TabletReplica>::~scoped_refptr() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/gutil/ref_counted.h:266:13 (libitest_util.so+0x1001ba)
#16 std::__1::allocator<scoped_refptr<kudu::tablet::TabletReplica> >::destroy(scoped_refptr<kudu::tablet::TabletReplica>*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:1811:92 (libmaster.so+0x3bc669)
#17 void std::__1::allocator_traits<std::__1::allocator<scoped_refptr<kudu::tablet::TabletReplica> > >::__destroy<scoped_refptr<kudu::tablet::TabletReplica> >(std::__1::integral_constant<bool, true>, std::__1::allocator<scoped_refptr<kudu::tablet::TabletReplica> >&, scoped_refptr<kudu::tablet::TabletReplica>*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:1703:21 (libmaster.so+0x3bc640)
#18 void std::__1::allocator_traits<std::__1::allocator<scoped_refptr<kudu::tablet::TabletReplica> > >::destroy<scoped_refptr<kudu::tablet::TabletReplica> >(std::__1::allocator<scoped_refptr<kudu::tablet::TabletReplica> >&, scoped_refptr<kudu::tablet::TabletReplica>*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:1544:14 (libmaster.so+0x3bc600)
#19 std::__1::__vector_base<scoped_refptr<kudu::tablet::TabletReplica>, std::__1::allocator<scoped_refptr<kudu::tablet::TabletReplica> > >::__destruct_at_end(scoped_refptr<kudu::tablet::TabletReplica>*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/vector:428:9 (libtserver.so+0x2b1096)
#20 std::__1::__vector_base<scoped_refptr<kudu::tablet::TabletReplica>, std::__1::allocator<scoped_refptr<kudu::tablet::TabletReplica> > >::clear() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/vector:371:29 (libtserver.so+0x2b0fb4)
#21 std::__1::__vector_base<scoped_refptr<kudu::tablet::TabletReplica>, std::__1::allocator<scoped_refptr<kudu::tablet::TabletReplica> > >::~__vector_base() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/vector:465:9 (libtserver.so+0x2b0d6b)
#22 std::__1::vector<scoped_refptr<kudu::tablet::TabletReplica>, std::__1::allocator<scoped_refptr<kudu::tablet::TabletReplica> > >::~vector() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/vector:557:5 (libtserver.so+0x289451)
#23 kudu::tserver::TSTabletManager::Shutdown() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/ts_tablet_manager.cc:1572:1 (libtserver.so+0x2b43be)
#24 kudu::tserver::TabletServer::ShutdownImpl() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/tablet_server.cc:189:22 (libtserver.so+0x25602e)
#25 kudu::tserver::TabletServer::Shutdown() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/tablet_server.h:67:5 (libtserver.so+0x25b7f9)
#26 kudu::tserver::MiniTabletServer::Shutdown() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/mini_tablet_server.cc:124:14 (libtserver.so+0x3135a2)
#27 kudu::itest::TServerQuiescingITest_TestDoesntAllowNewScansLeadersOnly_Test::TestBody() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/tablet_server_quiescing-itest.cc:308:7 (tablet_server_quiescing-itest+0x37166c)
#28 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x64e2f)
#29 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x64e2f)
#30 testing::Test::Run() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2674:5 (libgtest.so.1.12.1+0x42a31)
#31 testing::TestInfo::Run() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2853:11 (libgtest.so.1.12.1+0x43d48)
#32 testing::TestSuite::Run() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:3012:30 (libgtest.so.1.12.1+0x44d24)
#33 testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5870:44 (libgtest.so.1.12.1+0x59814)
#34 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x65cef)
#35 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x65cef)
#36 testing::UnitTest::Run() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5444:10 (libgtest.so.1.12.1+0x58dcc)
#37 RUN_ALL_TESTS() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/gtest/gtest.h:2293:73 (tablet_server_quiescing-itest+0x3b91bb)
#38 main /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/test_main.cc:115:10 (tablet_server_quiescing-itest+0x3b7dfc)
Previous read of size 8 at 0x7b4400038d48 by thread T343 (mutexes: write M447399201263932528):
#0 std::__1::vector<kudu::MemTracker*, std::__1::allocator<kudu::MemTracker*> >::end() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/vector:1536:30 (libkudu_util.so+0x3a14fa)
#1 kudu::MemTracker::Release(long) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/mem_tracker.cc:284:22 (libkudu_util.so+0x39ebd6)
#2 kudu::tablet::OpTracker::Release(kudu::tablet::OpDriver*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_tracker.cc:239:19 (libtablet.so+0x41d2c0)
#3 kudu::tablet::OpDriver::Finalize() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_driver.cc:606:16 (libtablet.so+0x416322)
#4 kudu::tablet::OpDriver::ApplyTask() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_driver.cc:577:5 (libtablet.so+0x415ab3)
#5 kudu::tablet::OpDriver::ApplyAsync()::$_2::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_driver.cc:524:65 (libtablet.so+0x419cc1)
#6 decltype(std::__1::forward<kudu::tablet::OpDriver::ApplyAsync()::$_2&>(fp)()) std::__1::__invoke<kudu::tablet::OpDriver::ApplyAsync()::$_2&>(kudu::tablet::OpDriver::ApplyAsync()::$_2&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libtablet.so+0x419c79)
#7 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::tablet::OpDriver::ApplyAsync()::$_2&>(kudu::tablet::OpDriver::ApplyAsync()::$_2&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libtablet.so+0x419c09)
#8 std::__1::__function::__alloc_func<kudu::tablet::OpDriver::ApplyAsync()::$_2, std::__1::allocator<kudu::tablet::OpDriver::ApplyAsync()::$_2>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libtablet.so+0x419bd1)
#9 std::__1::__function::__func<kudu::tablet::OpDriver::ApplyAsync()::$_2, std::__1::allocator<kudu::tablet::OpDriver::ApplyAsync()::$_2>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libtablet.so+0x418dfd)
#10 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (tablet_server_quiescing-itest+0x3b9454)
#11 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (tablet_server_quiescing-itest+0x3b9299)
#12 kudu::ThreadPool::DispatchThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:733:7 (libkudu_util.so+0x463464)
#13 kudu::ThreadPool::CreateThread()::$_1::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:815:48 (libkudu_util.so+0x4657b1)
#14 decltype(std::__1::forward<kudu::ThreadPool::CreateThread()::$_1&>(fp)()) std::__1::__invoke<kudu::ThreadPool::CreateThread()::$_1&>(kudu::ThreadPool::CreateThread()::$_1&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libkudu_util.so+0x465769)
#15 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::ThreadPool::CreateThread()::$_1&>(kudu::ThreadPool::CreateThread()::$_1&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libkudu_util.so+0x4656f9)
#16 std::__1::__function::__alloc_func<kudu::ThreadPool::CreateThread()::$_1, std::__1::allocator<kudu::ThreadPool::CreateThread()::$_1>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libkudu_util.so+0x4656c1)
#17 std::__1::__function::__func<kudu::ThreadPool::CreateThread()::$_1, std::__1::allocator<kudu::ThreadPool::CreateThread()::$_1>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libkudu_util.so+0x4649bd)
#18 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (tablet_server_quiescing-itest+0x3b9454)
#19 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (tablet_server_quiescing-itest+0x3b9299)
#20 kudu::Thread::SuperviseThread(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:694:3 (libkudu_util.so+0x44676a)
As if synchronized via sleep:
#0 nanosleep /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:363 (tablet_server_quiescing-itest+0x314a12)
#1 base::SleepForNanoseconds(long) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/gutil/sysinfo.cc:96:10 (libgutil.so+0xb5f0a)
#2 kudu::SleepFor(kudu::MonoDelta const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/monotime.cc:264:3 (libkudu_util.so+0x3d7f96)
#3 kudu::tablet::OpTracker::WaitForAllToFinish(kudu::MonoDelta const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_tracker.cc:298:5 (libtablet.so+0x41dba0)
#4 kudu::tablet::OpTracker::WaitForAllToFinish() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_tracker.cc:260:3 (libtablet.so+0x41d57e)
#5 kudu::tablet::TabletReplica::Stop() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet_replica.cc:350:17 (libtablet.so+0x3d13e7)
#6 kudu::tablet::TabletReplica::Shutdown() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet_replica.cc:380:3 (libtablet.so+0x3d1a2e)
#7 kudu::tserver::TSTabletManager::Shutdown() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/ts_tablet_manager.cc:1545:14 (libtserver.so+0x2b42b7)
#8 kudu::tserver::TabletServer::ShutdownImpl() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/tablet_server.cc:189:22 (libtserver.so+0x25602e)
#9 kudu::tserver::TabletServer::Shutdown() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/tablet_server.h:67:5 (libtserver.so+0x25b7f9)
#10 kudu::tserver::MiniTabletServer::Shutdown() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/mini_tablet_server.cc:124:14 (libtserver.so+0x3135a2)
#11 kudu::itest::TServerQuiescingITest_TestDoesntAllowNewScansLeadersOnly_Test::TestBody() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/tablet_server_quiescing-itest.cc:308:7 (tablet_server_quiescing-itest+0x37166c)
#12 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x64e2f)
#13 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x64e2f)
#14 testing::Test::Run() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2674:5 (libgtest.so.1.12.1+0x42a31)
#15 testing::TestInfo::Run() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2853:11 (libgtest.so.1.12.1+0x43d48)
#16 testing::TestSuite::Run() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:3012:30 (libgtest.so.1.12.1+0x44d24)
#17 testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5870:44 (libgtest.so.1.12.1+0x59814)
#18 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x65cef)
#19 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x65cef)
#20 testing::UnitTest::Run() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5444:10 (libgtest.so.1.12.1+0x58dcc)
#21 RUN_ALL_TESTS() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/gtest/gtest.h:2293:73 (tablet_server_quiescing-itest+0x3b91bb)
#22 main /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/test_main.cc:115:10 (tablet_server_quiescing-itest+0x3b7dfc)
Location is heap block of size 264 at 0x7b4400038cc0 allocated by thread T264:
#0 operator new(unsigned long) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cpp:64 (tablet_server_quiescing-itest+0x36eed7)
#1 std::__1::__libcpp_allocate(unsigned long, unsigned long) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/new:253:10 (libkudu_util.so+0x29bbf6)
#2 std::__1::allocator<std::__1::__shared_ptr_emplace<std::__1::shared_ptr<kudu::MemTracker> enable_make_shared<kudu::MemTracker>::make_shared<long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<kudu::MemTracker>&>(long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<kudu::MemTracker>&)::make_shared_enabler, std::__1::allocator<std::__1::shared_ptr<kudu::MemTracker> enable_make_shared<kudu::MemTracker>::make_shared<long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<kudu::MemTracker>&>(long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<kudu::MemTracker>&)::make_shared_enabler> > >::allocate(unsigned long) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:1789:34 (libkudu_util.so+0x3a6d35)
#3 std::__1::enable_if<!(is_array<long&>::value), std::__1::shared_ptr<long&> >::type std::__1::make_shared<std::__1::shared_ptr<kudu::MemTracker> enable_make_shared<kudu::MemTracker>::make_shared<long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<kudu::MemTracker>&>(long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<kudu::MemTracker>&)::make_shared_enabler, long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<kudu::MemTracker>&>(std::__1::shared_ptr<kudu::MemTracker> enable_make_shared<kudu::MemTracker>::make_shared<long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<kudu::MemTracker>&>(long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<kudu::MemTracker>&)::make_shared_enabler&&...) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:4290:45 (libkudu_util.so+0x3a6b11)
#4 std::__1::shared_ptr<kudu::MemTracker> enable_make_shared<kudu::MemTracker>::make_shared<long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<kudu::MemTracker>&>(long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<kudu::MemTracker>&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/make_shared.h:61:12 (libkudu_util.so+0x3a0445)
#5 kudu::MemTracker::CreateTracker(long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<kudu::MemTracker>) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/mem_tracker.cc:54:34 (libkudu_util.so+0x39e46e)
#6 kudu::tablet::OpTracker::StartMemoryTracking(std::__1::shared_ptr<kudu::MemTracker> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_tracker.cc:311:20 (libtablet.so+0x41dd7b)
#7 kudu::tablet::TabletReplica::Start(kudu::consensus::ConsensusBootstrapInfo const&, std::__1::shared_ptr<kudu::tablet::Tablet>, kudu::clock::Clock*, std::__1::shared_ptr<kudu::rpc::Messenger>, scoped_refptr<kudu::rpc::ResultTracker>, scoped_refptr<kudu::log::Log>, kudu::ThreadPool*, kudu::DnsResolver*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet_replica.cc:277:19 (libtablet.so+0x3d0736)
#8 kudu::tserver::TSTabletManager::OpenTablet(scoped_refptr<kudu::tablet::TabletReplica> const&, scoped_refptr<kudu::tserver::TransitionInProgressDeleter> const&, std::__1::atomic<int>*, std::__1::atomic<int>*, kudu::Timer*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/ts_tablet_manager.cc:1430:18 (libtserver.so+0x2b9b90)
#9 kudu::tserver::TSTabletManager::CreateNewTablet(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Partition const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Schema const&, kudu::PartitionSchema const&, kudu::consensus::RaftConfigPB, std::__1::optional<kudu::TableExtraConfigPB>, std::__1::optional<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::optional<kudu::TableTypePB>, scoped_refptr<kudu::tablet::TabletReplica>*)::$_11::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/ts_tablet_manager.cc:703:3 (libtserver.so+0x2cdb01)
#10 decltype(std::__1::forward<kudu::tserver::TSTabletManager::CreateNewTablet(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Partition const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Schema const&, kudu::PartitionSchema const&, kudu::consensus::RaftConfigPB, std::__1::optional<kudu::TableExtraConfigPB>, std::__1::optional<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::optional<kudu::TableTypePB>, scoped_refptr<kudu::tablet::TabletReplica>*)::$_11&>(fp)()) std::__1::__invoke<kudu::tserver::TSTabletManager::CreateNewTablet(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Partition const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Schema const&, kudu::PartitionSchema const&, kudu::consensus::RaftConfigPB, std::__1::optional<kudu::TableExtraConfigPB>, std::__1::optional<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::optional<kudu::TableTypePB>, scoped_refptr<kudu::tablet::TabletReplica>*)::$_11&>(kudu::tserver::TSTabletManager::CreateNewTablet(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Partition const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Schema const&, kudu::PartitionSchema const&, kudu::consensus::RaftConfigPB, std::__1::optional<kudu::TableExtraConfigPB>, std::__1::optional<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::optional<kudu::TableTypePB>, scoped_refptr<kudu::tablet::TabletReplica>*)::$_11&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libtserver.so+0x2cdaa9)
#11 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::tserver::TSTabletManager::CreateNewTablet(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Partition const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Schema const&, kudu::PartitionSchema const&, kudu::consensus::RaftConfigPB, std::__1::optional<kudu::TableExtraConfigPB>, std::__1::optional<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::optional<kudu::TableTypePB>, scoped_refptr<kudu::tablet::TabletReplica>*)::$_11&>(kudu::tserver::TSTabletManager::CreateNewTablet(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Partition const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Schema const&, kudu::PartitionSchema const&, kudu::consensus::RaftConfigPB, std::__1::optional<kudu::TableExtraConfigPB>, std::__1::optional<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::optional<kudu::TableTypePB>, scoped_refptr<kudu::tablet::TabletReplica>*)::$_11&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libtserver.so+0x2cda39)
#12 std::__1::__function::__alloc_func<kudu::tserver::TSTabletManager::CreateNewTablet(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Partition const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Schema const&, kudu::PartitionSchema const&, kudu::consensus::RaftConfigPB, std::__1::optional<kudu::TableExtraConfigPB>, std::__1::optional<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::optional<kudu::TableTypePB>, scoped_refptr<kudu::tablet::TabletReplica>*)::$_11, std::__1::allocator<kudu::tserver::TSTabletManager::CreateNewTablet(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Partition const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Schema const&, kudu::PartitionSchema const&, kudu::consensus::RaftConfigPB, std::__1::optional<kudu::TableExtraConfigPB>, std::__1::optional<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::optional<kudu::TableTypePB>, scoped_refptr<kudu::tablet::TabletReplica>*)::$_11>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libtserver.so+0x2cda01)
#13 std::__1::__function::__func<kudu::tserver::TSTabletManager::CreateNewTablet(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Partition const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Schema const&, kudu::PartitionSchema const&, kudu::consensus::RaftConfigPB, std::__1::optional<kudu::TableExtraConfigPB>, std::__1::optional<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::optional<kudu::TableTypePB>, scoped_refptr<kudu::tablet::TabletReplica>*)::$_11, std::__1::allocator<kudu::tserver::TSTabletManager::CreateNewTablet(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Partition const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Schema const&, kudu::PartitionSchema const&, kudu::consensus::RaftConfigPB, std::__1::optional<kudu::TableExtraConfigPB>, std::__1::optional<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::optional<kudu::TableTypePB>, scoped_refptr<kudu::tablet::TabletReplica>*)::$_11>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libtserver.so+0x2cccdd)
#14 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (tablet_server_quiescing-itest+0x3b9454)
#15 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (tablet_server_quiescing-itest+0x3b9299)
#16 kudu::ThreadPool::DispatchThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:733:7 (libkudu_util.so+0x463464)
#17 kudu::ThreadPool::CreateThread()::$_1::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:815:48 (libkudu_util.so+0x4657b1)
#18 decltype(std::__1::forward<kudu::ThreadPool::CreateThread()::$_1&>(fp)()) std::__1::__invoke<kudu::ThreadPool::CreateThread()::$_1&>(kudu::ThreadPool::CreateThread()::$_1&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libkudu_util.so+0x465769)
#19 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::ThreadPool::CreateThread()::$_1&>(kudu::ThreadPool::CreateThread()::$_1&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libkudu_util.so+0x4656f9)
#20 std::__1::__function::__alloc_func<kudu::ThreadPool::CreateThread()::$_1, std::__1::allocator<kudu::ThreadPool::CreateThread()::$_1>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libkudu_util.so+0x4656c1)
#21 std::__1::__function::__func<kudu::ThreadPool::CreateThread()::$_1, std::__1::allocator<kudu::ThreadPool::CreateThread()::$_1>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libkudu_util.so+0x4649bd)
#22 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (tablet_server_quiescing-itest+0x3b9454)
#23 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (tablet_server_quiescing-itest+0x3b9299)
#24 kudu::Thread::SuperviseThread(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:694:3 (libkudu_util.so+0x44676a)
Mutex M447399201263932528 is already destroyed.
Thread T343 'apply [worker]-' (tid=12500, running) created by thread T108 at:
#0 pthread_create /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:966 (tablet_server_quiescing-itest+0x2f34e5)
#1 kudu::Thread::StartThread(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<void ()>, unsigned long, scoped_refptr<kudu::Thread>*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:643:15 (libkudu_util.so+0x446018)
#2 kudu::Thread::Create(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<void ()>, scoped_refptr<kudu::Thread>*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.h:147:12 (libmaster.so+0x2d4b89)
#3 kudu::ThreadPool::CreateThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:814:10 (libkudu_util.so+0x46269b)
#4 kudu::ThreadPool::DoSubmit(std::__1::function<void ()>, kudu::ThreadPoolToken*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:599:21 (libkudu_util.so+0x46093f)
#5 kudu::ThreadPool::Submit(std::__1::function<void ()>) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:491:10 (libkudu_util.so+0x462c02)
#6 kudu::tablet::OpDriver::ApplyAsync() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_driver.cc:524:23 (libtablet.so+0x414e50)
#7 kudu::tablet::OpDriver::ReplicationFinished(kudu::Status const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_driver.cc:474:5 (libtablet.so+0x4153e9)
#8 kudu::tablet::OpDriver::Init(std::__1::unique_ptr<kudu::tablet::Op, std::__1::default_delete<kudu::tablet::Op> >, kudu::consensus::DriverType)::$_0::operator()(kudu::Status const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_driver.cc:186:21 (libtablet.so+0x41767a)
#9 decltype(std::__1::forward<kudu::tablet::OpDriver::Init(std::__1::unique_ptr<kudu::tablet::Op, std::__1::default_delete<kudu::tablet::Op> >, kudu::consensus::DriverType)::$_0&>(fp)(std::__1::forward<kudu::Status const&>(fp0))) std::__1::__invoke<kudu::tablet::OpDriver::Init(std::__1::unique_ptr<kudu::tablet::Op, std::__1::default_delete<kudu::tablet::Op> >, kudu::consensus::DriverType)::$_0&, kudu::Status const&>(kudu::tablet::OpDriver::Init(std::__1::unique_ptr<kudu::tablet::Op, std::__1::default_delete<kudu::tablet::Op> >, kudu::consensus::DriverType)::$_0&, kudu::Status const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libtablet.so+0x417608)
#10 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::tablet::OpDriver::Init(std::__1::unique_ptr<kudu::tablet::Op, std::__1::default_delete<kudu::tablet::Op> >, kudu::consensus::DriverType)::$_0&, kudu::Status const&>(kudu::tablet::OpDriver::Init(std::__1::unique_ptr<kudu::tablet::Op, std::__1::default_delete<kudu::tablet::Op> >, kudu::consensus::DriverType)::$_0&, kudu::Status const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libtablet.so+0x417588)
#11 std::__1::__function::__alloc_func<kudu::tablet::OpDriver::Init(std::__1::unique_ptr<kudu::tablet::Op, std::__1::default_delete<kudu::tablet::Op> >, kudu::consensus::DriverType)::$_0, std::__1::allocator<kudu::tablet::OpDriver::Init(std::__1::unique_ptr<kudu::tablet::Op, std::__1::default_delete<kudu::tablet::Op> >, kudu::consensus::DriverType)::$_0>, void (kudu::Status const&)>::operator()(kudu::Status const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libtablet.so+0x417543)
#12 std::__1::__function::__func<kudu::tablet::OpDriver::Init(std::__1::unique_ptr<kudu::tablet::Op, std::__1::default_delete<kudu::tablet::Op> >, kudu::consensus::DriverType)::$_0, std::__1::allocator<kudu::tablet::OpDriver::Init(std::__1::unique_ptr<kudu::tablet::Op, std::__1::default_delete<kudu::tablet::Op> >, kudu::consensus::DriverType)::$_0>, void (kudu::Status const&)>::operator()(kudu::Status const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libtablet.so+0x41675c)
#13 std::__1::__function::__value_func<void (kudu::Status const&)>::operator()(kudu::Status const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (libmaster.so+0x3ea336)
#14 std::__1::function<void (kudu::Status const&)>::operator()(kudu::Status const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (libmaster.so+0x3e4b88)
#15 kudu::consensus::ConsensusRound::NotifyReplicationFinished(kudu::Status const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/raft_consensus.cc:3315:3 (libconsensus.so+0x1aa2c0)
#16 kudu::consensus::PendingRounds::AdvanceCommittedIndex(long) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/pending_rounds.cc:195:12 (libconsensus.so+0x1819a3)
#17 kudu::consensus::RaftConsensus::NotifyCommitIndex(long) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/raft_consensus.cc:914:3 (libconsensus.so+0x19b294)
#18 kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const::'lambda'(kudu::consensus::PeerMessageQueueObserver*)::operator()(kudu::consensus::PeerMessageQueueObserver*) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_queue.cc:1493:3 (libconsensus.so+0x138644)
#19 decltype(std::__1::forward<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const::'lambda'(kudu::consensus::PeerMessageQueueObserver*)&>(fp)(std::__1::forward<kudu::consensus::PeerMessageQueueObserver*>(fp0))) std::__1::__invoke<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const::'lambda'(kudu::consensus::PeerMessageQueueObserver*)&, kudu::consensus::PeerMessageQueueObserver*>(kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const::'lambda'(kudu::consensus::PeerMessageQueueObserver*)&, kudu::consensus::PeerMessageQueueObserver*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libconsensus.so+0x1385f3)
#20 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const::'lambda'(kudu::consensus::PeerMessageQueueObserver*)&, kudu::consensus::PeerMessageQueueObserver*>(kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const::'lambda'(kudu::consensus::PeerMessageQueueObserver*)&, kudu::consensus::PeerMessageQueueObserver*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libconsensus.so+0x138568)
#21 std::__1::__function::__alloc_func<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const::'lambda'(kudu::consensus::PeerMessageQueueObserver*), std::__1::allocator<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const::'lambda'(kudu::consensus::PeerMessageQueueObserver*)>, void (kudu::consensus::PeerMessageQueueObserver*)>::operator()(kudu::consensus::PeerMessageQueueObserver*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libconsensus.so+0x138523)
#22 std::__1::__function::__func<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const::'lambda'(kudu::consensus::PeerMessageQueueObserver*), std::__1::allocator<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const::'lambda'(kudu::consensus::PeerMessageQueueObserver*)>, void (kudu::consensus::PeerMessageQueueObserver*)>::operator()(kudu::consensus::PeerMessageQueueObserver*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libconsensus.so+0x13780c)
#23 std::__1::__function::__value_func<void (kudu::consensus::PeerMessageQueueObserver*)>::operator()(kudu::consensus::PeerMessageQueueObserver*&&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (libconsensus.so+0x1614a6)
#24 std::__1::function<void (kudu::consensus::PeerMessageQueueObserver*)>::operator()(kudu::consensus::PeerMessageQueueObserver*) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (libconsensus.so+0x14946b)
#25 kudu::consensus::PeerMessageQueue::NotifyObserversTask(std::__1::function<void (kudu::consensus::PeerMessageQueueObserver*)> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_queue.cc:1564:5 (libconsensus.so+0x133acd)
#26 kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_queue.cc:1493:3 (libconsensus.so+0x1374a5)
#27 decltype(std::__1::forward<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3&>(fp)()) std::__1::__invoke<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3&>(kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libconsensus.so+0x137439)
#28 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3&>(kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libconsensus.so+0x1373c9)
#29 std::__1::__function::__alloc_func<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3, std::__1::allocator<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libconsensus.so+0x137391)
#30 std::__1::__function::__func<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3, std::__1::allocator<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libconsensus.so+0x13668d)
#31 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (tablet_server_quiescing-itest+0x3b9454)
#32 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (tablet_server_quiescing-itest+0x3b9299)
#33 kudu::ThreadPool::DispatchThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:733:7 (libkudu_util.so+0x463464)
#34 kudu::ThreadPool::CreateThread()::$_1::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:815:48 (libkudu_util.so+0x4657b1)
#35 decltype(std::__1::forward<kudu::ThreadPool::CreateThread()::$_1&>(fp)()) std::__1::__invoke<kudu::ThreadPool::CreateThread()::$_1&>(kudu::ThreadPool::CreateThread()::$_1&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libkudu_util.so+0x465769)
#36 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::ThreadPool::CreateThread()::$_1&>(kudu::ThreadPool::CreateThread()::$_1&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libkudu_util.so+0x4656f9)
#37 std::__1::__function::__alloc_func<kudu::ThreadPool::CreateThread()::$_1, std::__1::allocator<kudu::ThreadPool::CreateThread()::$_1>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libkudu_util.so+0x4656c1)
#38 std::__1::__function::__func<kudu::ThreadPool::CreateThread()::$_1, std::__1::allocator<kudu::ThreadPool::CreateThread()::$_1>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libkudu_util.so+0x4649bd)
#39 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (tablet_server_quiescing-itest+0x3b9454)
#40 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (tablet_server_quiescing-itest+0x3b9299)
#41 kudu::Thread::SuperviseThread(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:694:3 (libkudu_util.so+0x44676a)
Thread T264 'tablet-open [wo' (tid=12474, finished) created by thread T76 at:
#0 pthread_create /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:966 (tablet_server_quiescing-itest+0x2f34e5)
#1 kudu::Thread::StartThread(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<void ()>, unsigned long, scoped_refptr<kudu::Thread>*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:643:15 (libkudu_util.so+0x446018)
#2 kudu::Thread::Create(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<void ()>, scoped_refptr<kudu::Thread>*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.h:147:12 (libmaster.so+0x2d4b89)
#3 kudu::ThreadPool::CreateThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:814:10 (libkudu_util.so+0x46269b)
#4 kudu::ThreadPool::DoSubmit(std::__1::function<void ()>, kudu::ThreadPoolToken*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:599:21 (libkudu_util.so+0x46093f)
#5 kudu::ThreadPool::Submit(std::__1::function<void ()>) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:491:10 (libkudu_util.so+0x462c02)
#6 kudu::tserver::TSTabletManager::CreateNewTablet(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Partition const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, kudu::Schema const&, kudu::PartitionSchema const&, kudu::consensus::RaftConfigPB, std::__1::optional<kudu::TableExtraConfigPB>, std::__1::optional<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > >, std::__1::optional<kudu::TableTypePB>, scoped_refptr<kudu::tablet::TabletReplica>*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/ts_tablet_manager.cc:703:3 (libtserver.so+0x2b6d82)
#7 kudu::tserver::TabletServiceAdminImpl::CreateTablet(kudu::tserver::CreateTabletRequestPB const*, kudu::tserver::CreateTabletResponsePB*, kudu::rpc::RpcContext*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/tablet_service.cc:1475:34 (libtserver.so+0x267209)
#8 kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_1::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /home/jenkins-slave/workspace/build_and_test_flaky@2/build/tsan/src/kudu/tserver/tserver_admin.service.cc:189:13 (libtserver_admin_proto.so+0x68bf4)
#9 decltype(std::__1::forward<kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_1&>(fp)(std::__1::forward<google::protobuf::Message const*>(fp0), std::__1::forward<google::protobuf::Message*>(fp0), std::__1::forward<kudu::rpc::RpcContext*>(fp0))) std::__1::__invoke<kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_1&, google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*>(kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_1&, google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libtserver_admin_proto.so+0x68b82)
#10 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_1&, google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*>(kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_1&, google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libtserver_admin_proto.so+0x68ab1)
#11 std::__1::__function::__alloc_func<kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_1, std::__1::allocator<kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_1>, void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libtserver_admin_proto.so+0x68a2c)
#12 std::__1::__function::__func<kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_1, std::__1::allocator<kudu::tserver::TabletServerAdminServiceIf::TabletServerAdminServiceIf(scoped_refptr<kudu::MetricEntity> const&, scoped_refptr<kudu::rpc::ResultTracker> const&)::$_1>, void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libtserver_admin_proto.so+0x67ce2)
#13 std::__1::__function::__value_func<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*&&, google::protobuf::Message*&&, kudu::rpc::RpcContext*&&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (libkrpc.so+0x1f3ddc)
#14 std::__1::function<void (google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*)>::operator()(google::protobuf::Message const*, google::protobuf::Message*, kudu::rpc::RpcContext*) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (libkrpc.so+0x1f3216)
#15 kudu::rpc::GeneratedServiceIf::Handle(kudu::rpc::InboundCall*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/service_if.cc:137:3 (libkrpc.so+0x1f2bbf)
#16 kudu::rpc::ServicePool::RunThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/service_pool.cc:229:15 (libkrpc.so+0x1f5ed3)
#17 kudu::rpc::ServicePool::Init(int)::$_0::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/service_pool.cc:92:5 (libkrpc.so+0x1f71f1)
#18 decltype(std::__1::forward<kudu::rpc::ServicePool::Init(int)::$_0&>(fp)()) std::__1::__invoke<kudu::rpc::ServicePool::Init(int)::$_0&>(kudu::rpc::ServicePool::Init(int)::$_0&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libkrpc.so+0x1f71a9)
#19 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::rpc::ServicePool::Init(int)::$_0&>(kudu::rpc::ServicePool::Init(int)::$_0&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libkrpc.so+0x1f7139)
#20 std::__1::__function::__alloc_func<kudu::rpc::ServicePool::Init(int)::$_0, std::__1::allocator<kudu::rpc::ServicePool::Init(int)::$_0>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libkrpc.so+0x1f7101)
#21 std::__1::__function::__func<kudu::rpc::ServicePool::Init(int)::$_0, std::__1::allocator<kudu::rpc::ServicePool::Init(int)::$_0>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libkrpc.so+0x1f63fd)
#22 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (tablet_server_quiescing-itest+0x3b9454)
#23 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (tablet_server_quiescing-itest+0x3b9299)
#24 kudu::Thread::SuperviseThread(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:694:3 (libkudu_util.so+0x44676a)
SUMMARY: ThreadSanitizer: data race /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/vector:429:12 in std::__1::__vector_base<kudu::MemTracker*, std::__1::allocator<kudu::MemTracker*> >::__destruct_at_end(kudu::MemTracker**)
==================
==================
WARNING: ThreadSanitizer: data race (pid=10634)
Write of size 8 at 0x7b08001019c0 by main thread:
#0 operator delete(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cpp:126 (tablet_server_quiescing-itest+0x36f7a9)
#1 std::__1::_DeallocateCaller::__do_call(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/new:334:12 (tablet_server_quiescing-itest+0x396589)
#2 std::__1::_DeallocateCaller::__do_deallocate_handle_size(void*, unsigned long) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/new:292:12 (tablet_server_quiescing-itest+0x396529)
#3 std::__1::_DeallocateCaller::__do_deallocate_handle_size_align(void*, unsigned long, unsigned long) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/new:268:14 (libkudu_util.so+0x29c392)
#4 std::__1::__libcpp_deallocate(void*, unsigned long, unsigned long) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/new:340:3 (libkudu_util.so+0x29c329)
#5 std::__1::allocator<kudu::MemTracker*>::deallocate(kudu::MemTracker**, unsigned long) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:1798:10 (libkudu_util.so+0x3a86fa)
#6 std::__1::allocator_traits<std::__1::allocator<kudu::MemTracker*> >::deallocate(std::__1::allocator<kudu::MemTracker*>&, kudu::MemTracker**, unsigned long) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:1533:14 (libkudu_util.so+0x3a8579)
#7 std::__1::__vector_base<kudu::MemTracker*, std::__1::allocator<kudu::MemTracker*> >::~__vector_base() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/vector:466:9 (libkudu_util.so+0x3a8357)
#8 std::__1::vector<kudu::MemTracker*, std::__1::allocator<kudu::MemTracker*> >::~vector() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/vector:557:5 (libkudu_util.so+0x3a05e1)
#9 kudu::MemTracker::~MemTracker() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/mem_tracker.cc:83:1 (libkudu_util.so+0x3a0193)
#10 std::__1::__shared_ptr_emplace<std::__1::shared_ptr<kudu::MemTracker> enable_make_shared<kudu::MemTracker>::make_shared<long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<kudu::MemTracker>&>(long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<kudu::MemTracker>&)::make_shared_enabler, std::__1::allocator<std::__1::shared_ptr<kudu::MemTracker> enable_make_shared<kudu::MemTracker>::make_shared<long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<kudu::MemTracker>&>(long&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<kudu::MemTracker>&)::make_shared_enabler> >::__on_zero_shared() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:3503:23 (libkudu_util.so+0x3a7455)
#11 std::__1::__shared_count::__release_shared() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:3341:9 (libtablet_test_util.so+0x35485)
#12 std::__1::__shared_weak_count::__release_shared() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:3383:27 (libtablet_test_util.so+0x35429)
#13 std::__1::shared_ptr<kudu::MemTracker>::~shared_ptr() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:4098:19 (libtablet_test_util.so+0x31c18)
#14 kudu::tablet::OpTracker::~OpTracker() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_tracker.cc:142:1 (libtablet.so+0x41c8f4)
#15 kudu::tablet::TabletReplica::~TabletReplica() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet_replica.cc:202:1 (libtablet.so+0x3cf6d5)
#16 kudu::tablet::TabletReplica::~TabletReplica() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet_replica.cc:197:33 (libtablet.so+0x3cf8e9)
#17 kudu::RefCountedThreadSafe<kudu::tablet::TabletReplica, kudu::DefaultRefCountedThreadSafeTraits<kudu::tablet::TabletReplica> >::DeleteInternal(kudu::tablet::TabletReplica const*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/gutil/ref_counted.h:153:44 (libitest_util.so+0x1002d7)
#18 kudu::DefaultRefCountedThreadSafeTraits<kudu::tablet::TabletReplica>::Destruct(kudu::tablet::TabletReplica const*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/gutil/ref_counted.h:116:5 (libitest_util.so+0x100299)
#19 kudu::RefCountedThreadSafe<kudu::tablet::TabletReplica, kudu::DefaultRefCountedThreadSafeTraits<kudu::tablet::TabletReplica> >::Release() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/gutil/ref_counted.h:144:7 (libitest_util.so+0x100259)
#20 scoped_refptr<kudu::tablet::TabletReplica>::~scoped_refptr() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/gutil/ref_counted.h:266:13 (libitest_util.so+0x1001ba)
#21 std::__1::allocator<scoped_refptr<kudu::tablet::TabletReplica> >::destroy(scoped_refptr<kudu::tablet::TabletReplica>*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:1811:92 (libmaster.so+0x3bc669)
#22 void std::__1::allocator_traits<std::__1::allocator<scoped_refptr<kudu::tablet::TabletReplica> > >::__destroy<scoped_refptr<kudu::tablet::TabletReplica> >(std::__1::integral_constant<bool, true>, std::__1::allocator<scoped_refptr<kudu::tablet::TabletReplica> >&, scoped_refptr<kudu::tablet::TabletReplica>*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:1703:21 (libmaster.so+0x3bc640)
#23 void std::__1::allocator_traits<std::__1::allocator<scoped_refptr<kudu::tablet::TabletReplica> > >::destroy<scoped_refptr<kudu::tablet::TabletReplica> >(std::__1::allocator<scoped_refptr<kudu::tablet::TabletReplica> >&, scoped_refptr<kudu::tablet::TabletReplica>*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/memory:1544:14 (libmaster.so+0x3bc600)
#24 std::__1::__vector_base<scoped_refptr<kudu::tablet::TabletReplica>, std::__1::allocator<scoped_refptr<kudu::tablet::TabletReplica> > >::__destruct_at_end(scoped_refptr<kudu::tablet::TabletReplica>*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/vector:428:9 (libtserver.so+0x2b1096)
#25 std::__1::__vector_base<scoped_refptr<kudu::tablet::TabletReplica>, std::__1::allocator<scoped_refptr<kudu::tablet::TabletReplica> > >::clear() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/vector:371:29 (libtserver.so+0x2b0fb4)
#26 std::__1::__vector_base<scoped_refptr<kudu::tablet::TabletReplica>, std::__1::allocator<scoped_refptr<kudu::tablet::TabletReplica> > >::~__vector_base() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/vector:465:9 (libtserver.so+0x2b0d6b)
#27 std::__1::vector<scoped_refptr<kudu::tablet::TabletReplica>, std::__1::allocator<scoped_refptr<kudu::tablet::TabletReplica> > >::~vector() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/vector:557:5 (libtserver.so+0x289451)
#28 kudu::tserver::TSTabletManager::Shutdown() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/ts_tablet_manager.cc:1572:1 (libtserver.so+0x2b43be)
#29 kudu::tserver::TabletServer::ShutdownImpl() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/tablet_server.cc:189:22 (libtserver.so+0x25602e)
#30 kudu::tserver::TabletServer::Shutdown() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/tablet_server.h:67:5 (libtserver.so+0x25b7f9)
#31 kudu::tserver::MiniTabletServer::Shutdown() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/mini_tablet_server.cc:124:14 (libtserver.so+0x3135a2)
#32 kudu::itest::TServerQuiescingITest_TestDoesntAllowNewScansLeadersOnly_Test::TestBody() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/tablet_server_quiescing-itest.cc:308:7 (tablet_server_quiescing-itest+0x37166c)
#33 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x64e2f)
#34 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x64e2f)
#35 testing::Test::Run() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2674:5 (libgtest.so.1.12.1+0x42a31)
#36 testing::TestInfo::Run() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2853:11 (libgtest.so.1.12.1+0x43d48)
#37 testing::TestSuite::Run() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:3012:30 (libgtest.so.1.12.1+0x44d24)
#38 testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5870:44 (libgtest.so.1.12.1+0x59814)
#39 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x65cef)
#40 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x65cef)
#41 testing::UnitTest::Run() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5444:10 (libgtest.so.1.12.1+0x58dcc)
#42 RUN_ALL_TESTS() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/gtest/gtest.h:2293:73 (tablet_server_quiescing-itest+0x3b91bb)
#43 main /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/test_main.cc:115:10 (tablet_server_quiescing-itest+0x3b7dfc)
Previous read of size 8 at 0x7b08001019c0 by thread T343 (mutexes: write M447399201263932528):
#0 kudu::MemTracker::Release(long) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/mem_tracker.cc:285:5 (libkudu_util.so+0x39ec12)
#1 kudu::tablet::OpTracker::Release(kudu::tablet::OpDriver*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_tracker.cc:239:19 (libtablet.so+0x41d2c0)
#2 kudu::tablet::OpDriver::Finalize() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_driver.cc:606:16 (libtablet.so+0x416322)
#3 kudu::tablet::OpDriver::ApplyTask() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_driver.cc:577:5 (libtablet.so+0x415ab3)
#4 kudu::tablet::OpDriver::ApplyAsync()::$_2::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_driver.cc:524:65 (libtablet.so+0x419cc1)
#5 decltype(std::__1::forward<kudu::tablet::OpDriver::ApplyAsync()::$_2&>(fp)()) std::__1::__invoke<kudu::tablet::OpDriver::ApplyAsync()::$_2&>(kudu::tablet::OpDriver::ApplyAsync()::$_2&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libtablet.so+0x419c79)
#6 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::tablet::OpDriver::ApplyAsync()::$_2&>(kudu::tablet::OpDriver::ApplyAsync()::$_2&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libtablet.so+0x419c09)
#7 std::__1::__function::__alloc_func<kudu::tablet::OpDriver::ApplyAsync()::$_2, std::__1::allocator<kudu::tablet::OpDriver::ApplyAsync()::$_2>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libtablet.so+0x419bd1)
#8 std::__1::__function::__func<kudu::tablet::OpDriver::ApplyAsync()::$_2, std::__1::allocator<kudu::tablet::OpDriver::ApplyAsync()::$_2>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libtablet.so+0x418dfd)
#9 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (tablet_server_quiescing-itest+0x3b9454)
#10 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (tablet_server_quiescing-itest+0x3b9299)
#11 kudu::ThreadPool::DispatchThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:733:7 (libkudu_util.so+0x463464)
#12 kudu::ThreadPool::CreateThread()::$_1::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:815:48 (libkudu_util.so+0x4657b1)
#13 decltype(std::__1::forward<kudu::ThreadPool::CreateThread()::$_1&>(fp)()) std::__1::__invoke<kudu::ThreadPool::CreateThread()::$_1&>(kudu::ThreadPool::CreateThread()::$_1&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libkudu_util.so+0x465769)
#14 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::ThreadPool::CreateThread()::$_1&>(kudu::ThreadPool::CreateThread()::$_1&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libkudu_util.so+0x4656f9)
#15 std::__1::__function::__alloc_func<kudu::ThreadPool::CreateThread()::$_1, std::__1::allocator<kudu::ThreadPool::CreateThread()::$_1>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libkudu_util.so+0x4656c1)
#16 std::__1::__function::__func<kudu::ThreadPool::CreateThread()::$_1, std::__1::allocator<kudu::ThreadPool::CreateThread()::$_1>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libkudu_util.so+0x4649bd)
#17 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (tablet_server_quiescing-itest+0x3b9454)
#18 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (tablet_server_quiescing-itest+0x3b9299)
#19 kudu::Thread::SuperviseThread(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:694:3 (libkudu_util.so+0x44676a)
As if synchronized via sleep:
#0 nanosleep /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:363 (tablet_server_quiescing-itest+0x314a12)
#1 base::SleepForNanoseconds(long) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/gutil/sysinfo.cc:96:10 (libgutil.so+0xb5f0a)
#2 kudu::SleepFor(kudu::MonoDelta const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/monotime.cc:264:3 (libkudu_util.so+0x3d7f96)
#3 kudu::tablet::OpTracker::WaitForAllToFinish(kudu::MonoDelta const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_tracker.cc:298:5 (libtablet.so+0x41dba0)
#4 kudu::tablet::OpTracker::WaitForAllToFinish() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_tracker.cc:260:3 (libtablet.so+0x41d57e)
#5 kudu::tablet::TabletReplica::Stop() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet_replica.cc:350:17 (libtablet.so+0x3d13e7)
#6 kudu::tablet::TabletReplica::Shutdown() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet_replica.cc:380:3 (libtablet.so+0x3d1a2e)
#7 kudu::tserver::TSTabletManager::Shutdown() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/ts_tablet_manager.cc:1545:14 (libtserver.so+0x2b42b7)
#8 kudu::tserver::TabletServer::ShutdownImpl() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/tablet_server.cc:189:22 (libtserver.so+0x25602e)
#9 kudu::tserver::TabletServer::Shutdown() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/tablet_server.h:67:5 (libtserver.so+0x25b7f9)
#10 kudu::tserver::MiniTabletServer::Shutdown() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/mini_tablet_server.cc:124:14 (libtserver.so+0x3135a2)
#11 kudu::itest::TServerQuiescingITest_TestDoesntAllowNewScansLeadersOnly_Test::TestBody() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/tablet_server_quiescing-itest.cc:308:7 (tablet_server_quiescing-itest+0x37166c)
#12 void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x64e2f)
#13 void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x64e2f)
#14 testing::Test::Run() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2674:5 (libgtest.so.1.12.1+0x42a31)
#15 testing::TestInfo::Run() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2853:11 (libgtest.so.1.12.1+0x43d48)
#16 testing::TestSuite::Run() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:3012:30 (libgtest.so.1.12.1+0x44d24)
#17 testing::internal::UnitTestImpl::RunAllTests() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5870:44 (libgtest.so.1.12.1+0x59814)
#18 bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2599:10 (libgtest.so.1.12.1+0x65cef)
#19 bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:2635:14 (libgtest.so.1.12.1+0x65cef)
#20 testing::UnitTest::Run() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/googletest-release-1.12.1/googletest/src/gtest.cc:5444:10 (libgtest.so.1.12.1+0x58dcc)
#21 RUN_ALL_TESTS() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/gtest/gtest.h:2293:73 (tablet_server_quiescing-itest+0x3b91bb)
#22 main /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/test_main.cc:115:10 (tablet_server_quiescing-itest+0x3b7dfc)
Mutex M447399201263932528 is already destroyed.
Thread T343 'apply [worker]-' (tid=12500, running) created by thread T108 at:
#0 pthread_create /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:966 (tablet_server_quiescing-itest+0x2f34e5)
#1 kudu::Thread::StartThread(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<void ()>, unsigned long, scoped_refptr<kudu::Thread>*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:643:15 (libkudu_util.so+0x446018)
#2 kudu::Thread::Create(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, std::__1::function<void ()>, scoped_refptr<kudu::Thread>*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.h:147:12 (libmaster.so+0x2d4b89)
#3 kudu::ThreadPool::CreateThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:814:10 (libkudu_util.so+0x46269b)
#4 kudu::ThreadPool::DoSubmit(std::__1::function<void ()>, kudu::ThreadPoolToken*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:599:21 (libkudu_util.so+0x46093f)
#5 kudu::ThreadPool::Submit(std::__1::function<void ()>) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:491:10 (libkudu_util.so+0x462c02)
#6 kudu::tablet::OpDriver::ApplyAsync() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_driver.cc:524:23 (libtablet.so+0x414e50)
#7 kudu::tablet::OpDriver::ReplicationFinished(kudu::Status const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_driver.cc:474:5 (libtablet.so+0x4153e9)
#8 kudu::tablet::OpDriver::Init(std::__1::unique_ptr<kudu::tablet::Op, std::__1::default_delete<kudu::tablet::Op> >, kudu::consensus::DriverType)::$_0::operator()(kudu::Status const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/ops/op_driver.cc:186:21 (libtablet.so+0x41767a)
#9 decltype(std::__1::forward<kudu::tablet::OpDriver::Init(std::__1::unique_ptr<kudu::tablet::Op, std::__1::default_delete<kudu::tablet::Op> >, kudu::consensus::DriverType)::$_0&>(fp)(std::__1::forward<kudu::Status const&>(fp0))) std::__1::__invoke<kudu::tablet::OpDriver::Init(std::__1::unique_ptr<kudu::tablet::Op, std::__1::default_delete<kudu::tablet::Op> >, kudu::consensus::DriverType)::$_0&, kudu::Status const&>(kudu::tablet::OpDriver::Init(std::__1::unique_ptr<kudu::tablet::Op, std::__1::default_delete<kudu::tablet::Op> >, kudu::consensus::DriverType)::$_0&, kudu::Status const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libtablet.so+0x417608)
#10 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::tablet::OpDriver::Init(std::__1::unique_ptr<kudu::tablet::Op, std::__1::default_delete<kudu::tablet::Op> >, kudu::consensus::DriverType)::$_0&, kudu::Status const&>(kudu::tablet::OpDriver::Init(std::__1::unique_ptr<kudu::tablet::Op, std::__1::default_delete<kudu::tablet::Op> >, kudu::consensus::DriverType)::$_0&, kudu::Status const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libtablet.so+0x417588)
#11 std::__1::__function::__alloc_func<kudu::tablet::OpDriver::Init(std::__1::unique_ptr<kudu::tablet::Op, std::__1::default_delete<kudu::tablet::Op> >, kudu::consensus::DriverType)::$_0, std::__1::allocator<kudu::tablet::OpDriver::Init(std::__1::unique_ptr<kudu::tablet::Op, std::__1::default_delete<kudu::tablet::Op> >, kudu::consensus::DriverType)::$_0>, void (kudu::Status const&)>::operator()(kudu::Status const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libtablet.so+0x417543)
#12 std::__1::__function::__func<kudu::tablet::OpDriver::Init(std::__1::unique_ptr<kudu::tablet::Op, std::__1::default_delete<kudu::tablet::Op> >, kudu::consensus::DriverType)::$_0, std::__1::allocator<kudu::tablet::OpDriver::Init(std::__1::unique_ptr<kudu::tablet::Op, std::__1::default_delete<kudu::tablet::Op> >, kudu::consensus::DriverType)::$_0>, void (kudu::Status const&)>::operator()(kudu::Status const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libtablet.so+0x41675c)
#13 std::__1::__function::__value_func<void (kudu::Status const&)>::operator()(kudu::Status const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (libmaster.so+0x3ea336)
#14 std::__1::function<void (kudu::Status const&)>::operator()(kudu::Status const&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (libmaster.so+0x3e4b88)
#15 kudu::consensus::ConsensusRound::NotifyReplicationFinished(kudu::Status const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/raft_consensus.cc:3315:3 (libconsensus.so+0x1aa2c0)
#16 kudu::consensus::PendingRounds::AdvanceCommittedIndex(long) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/pending_rounds.cc:195:12 (libconsensus.so+0x1819a3)
#17 kudu::consensus::RaftConsensus::NotifyCommitIndex(long) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/raft_consensus.cc:914:3 (libconsensus.so+0x19b294)
#18 kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const::'lambda'(kudu::consensus::PeerMessageQueueObserver*)::operator()(kudu::consensus::PeerMessageQueueObserver*) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_queue.cc:1493:3 (libconsensus.so+0x138644)
#19 decltype(std::__1::forward<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const::'lambda'(kudu::consensus::PeerMessageQueueObserver*)&>(fp)(std::__1::forward<kudu::consensus::PeerMessageQueueObserver*>(fp0))) std::__1::__invoke<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const::'lambda'(kudu::consensus::PeerMessageQueueObserver*)&, kudu::consensus::PeerMessageQueueObserver*>(kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const::'lambda'(kudu::consensus::PeerMessageQueueObserver*)&, kudu::consensus::PeerMessageQueueObserver*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libconsensus.so+0x1385f3)
#20 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const::'lambda'(kudu::consensus::PeerMessageQueueObserver*)&, kudu::consensus::PeerMessageQueueObserver*>(kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const::'lambda'(kudu::consensus::PeerMessageQueueObserver*)&, kudu::consensus::PeerMessageQueueObserver*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libconsensus.so+0x138568)
#21 std::__1::__function::__alloc_func<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const::'lambda'(kudu::consensus::PeerMessageQueueObserver*), std::__1::allocator<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const::'lambda'(kudu::consensus::PeerMessageQueueObserver*)>, void (kudu::consensus::PeerMessageQueueObserver*)>::operator()(kudu::consensus::PeerMessageQueueObserver*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libconsensus.so+0x138523)
#22 std::__1::__function::__func<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const::'lambda'(kudu::consensus::PeerMessageQueueObserver*), std::__1::allocator<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const::'lambda'(kudu::consensus::PeerMessageQueueObserver*)>, void (kudu::consensus::PeerMessageQueueObserver*)>::operator()(kudu::consensus::PeerMessageQueueObserver*&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libconsensus.so+0x13780c)
#23 std::__1::__function::__value_func<void (kudu::consensus::PeerMessageQueueObserver*)>::operator()(kudu::consensus::PeerMessageQueueObserver*&&) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (libconsensus.so+0x1614a6)
#24 std::__1::function<void (kudu::consensus::PeerMessageQueueObserver*)>::operator()(kudu::consensus::PeerMessageQueueObserver*) const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (libconsensus.so+0x14946b)
#25 kudu::consensus::PeerMessageQueue::NotifyObserversTask(std::__1::function<void (kudu::consensus::PeerMessageQueueObserver*)> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_queue.cc:1564:5 (libconsensus.so+0x133acd)
#26 kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_queue.cc:1493:3 (libconsensus.so+0x1374a5)
#27 decltype(std::__1::forward<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3&>(fp)()) std::__1::__invoke<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3&>(kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libconsensus.so+0x137439)
#28 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3&>(kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libconsensus.so+0x1373c9)
#29 std::__1::__function::__alloc_func<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3, std::__1::allocator<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libconsensus.so+0x137391)
#30 std::__1::__function::__func<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3, std::__1::allocator<kudu::consensus::PeerMessageQueue::NotifyObserversOfCommitIndexChange(long)::$_3>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libconsensus.so+0x13668d)
#31 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (tablet_server_quiescing-itest+0x3b9454)
#32 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (tablet_server_quiescing-itest+0x3b9299)
#33 kudu::ThreadPool::DispatchThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:733:7 (libkudu_util.so+0x463464)
#34 kudu::ThreadPool::CreateThread()::$_1::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:815:48 (libkudu_util.so+0x4657b1)
#35 decltype(std::__1::forward<kudu::ThreadPool::CreateThread()::$_1&>(fp)()) std::__1::__invoke<kudu::ThreadPool::CreateThread()::$_1&>(kudu::ThreadPool::CreateThread()::$_1&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/type_traits:3905:1 (libkudu_util.so+0x465769)
#36 void std::__1::__invoke_void_return_wrapper<void>::__call<kudu::ThreadPool::CreateThread()::$_1&>(kudu::ThreadPool::CreateThread()::$_1&) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/__functional_base:348:9 (libkudu_util.so+0x4656f9)
#37 std::__1::__function::__alloc_func<kudu::ThreadPool::CreateThread()::$_1, std::__1::allocator<kudu::ThreadPool::CreateThread()::$_1>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1557:16 (libkudu_util.so+0x4656c1)
#38 std::__1::__function::__func<kudu::ThreadPool::CreateThread()::$_1, std::__1::allocator<kudu::ThreadPool::CreateThread()::$_1>, void ()>::operator()() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1731:12 (libkudu_util.so+0x4649bd)
#39 std::__1::__function::__value_func<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:1884:16 (tablet_server_quiescing-itest+0x3b9454)
#40 std::__1::function<void ()>::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2556:12 (tablet_server_quiescing-itest+0x3b9299)
#41 kudu::Thread::SuperviseThread(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:694:3 (libkudu_util.so+0x44676a)
SUMMARY: ThreadSanitizer: data race /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_new_delete.cpp:126 in operator delete(void*)
==================
I20250904 13:57:36.473472 12486 meta_cache.cc:1510] marking tablet server 43cfdecc88a74213b3c25129a27e9a2c (127.10.98.131:40633) as failed
W20250904 13:57:36.474035 12486 meta_cache.cc:302] tablet eb9144f161da423f9fe899f96f54081c: replica 43cfdecc88a74213b3c25129a27e9a2c (127.10.98.131:40633) has failed: Network error: TS failed: recv got EOF from unknown peer (error 108)
I20250904 13:57:36.476632 12275 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "eb9144f161da423f9fe899f96f54081c" candidate_uuid: "724a4b4bc45048caa6e55b4db69ef398" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "464eb7dc213646849defa0776cfedb52"
I20250904 13:57:36.477092 12276 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "eb9144f161da423f9fe899f96f54081c" candidate_uuid: "724a4b4bc45048caa6e55b4db69ef398" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "464eb7dc213646849defa0776cfedb52"
I20250904 13:57:36.477456 12275 raft_consensus.cc:2391] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 724a4b4bc45048caa6e55b4db69ef398 in current term 3: Already voted for candidate 464eb7dc213646849defa0776cfedb52 in this term.
I20250904 13:57:36.479105 12316 leader_election.cc:304] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 724a4b4bc45048caa6e55b4db69ef398; no voters: 43cfdecc88a74213b3c25129a27e9a2c, 464eb7dc213646849defa0776cfedb52
W20250904 13:57:36.482105 12242 connection.cc:700] client connection to 127.10.98.131:40633 send error: Network error: sendmsg error: Broken pipe (error 32)
I20250904 13:57:36.482014 12316 leader_election.cc:304] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 724a4b4bc45048caa6e55b4db69ef398; no voters: 43cfdecc88a74213b3c25129a27e9a2c, 464eb7dc213646849defa0776cfedb52
I20250904 13:57:36.485193 10634 tablet_server.cc:195] TabletServer@127.10.98.131:0 shutdown complete.
I20250904 13:57:36.502027 12507 raft_consensus.cc:2747] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20250904 13:57:36.502789 12507 raft_consensus.cc:2747] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20250904 13:57:36.509099 12351 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "eb9144f161da423f9fe899f96f54081c" candidate_uuid: "464eb7dc213646849defa0776cfedb52" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "724a4b4bc45048caa6e55b4db69ef398"
W20250904 13:57:36.509200 12242 leader_election.cc:336] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 43cfdecc88a74213b3c25129a27e9a2c (127.10.98.131:40633): Network error: Client connection negotiation failed: client connection to 127.10.98.131:40633: connect: Connection refused (error 111)
I20250904 13:57:36.509328 12350 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "eb9144f161da423f9fe899f96f54081c" candidate_uuid: "464eb7dc213646849defa0776cfedb52" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "724a4b4bc45048caa6e55b4db69ef398"
W20250904 13:57:36.509922 12242 leader_election.cc:336] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 43cfdecc88a74213b3c25129a27e9a2c (127.10.98.131:40633): Network error: Client connection negotiation failed: client connection to 127.10.98.131:40633: connect: Connection refused (error 111)
I20250904 13:57:36.510005 12351 raft_consensus.cc:2366] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 464eb7dc213646849defa0776cfedb52 for earlier term 2. Current term is 3.
I20250904 13:57:36.510859 12242 leader_election.cc:304] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 464eb7dc213646849defa0776cfedb52; no voters: 43cfdecc88a74213b3c25129a27e9a2c, 724a4b4bc45048caa6e55b4db69ef398
I20250904 13:57:36.512353 12511 raft_consensus.cc:2747] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20250904 13:57:36.512533 12242 leader_election.cc:400] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [CANDIDATE]: Term 2 election: Vote denied by peer 724a4b4bc45048caa6e55b4db69ef398 (127.10.98.130:46811) with higher term. Message: Invalid argument: T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 464eb7dc213646849defa0776cfedb52 for earlier term 2. Current term is 3.
I20250904 13:57:36.512880 12242 leader_election.cc:403] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20250904 13:57:36.513335 12511 raft_consensus.cc:2747] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 724a4b4bc45048caa6e55b4db69ef398 (127.10.98.130:46811) with higher term. Message: Invalid argument: T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 464eb7dc213646849defa0776cfedb52 for earlier term 2. Current term is 3.
W20250904 13:57:36.539594 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.539863 12330 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.540011 12329 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.584697 12256 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.585095 12255 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.585501 12254 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.644392 12330 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.646322 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.647694 12330 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.671191 12255 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.671219 12254 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.676853 12254 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.731300 12330 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.731401 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.740761 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.760830 12254 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.763427 12254 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.773401 12254 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.826653 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.840190 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.860188 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.860888 12254 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.873370 12254 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.893402 12254 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.933153 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.952188 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.961802 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.968735 12254 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.989780 12486 scanner-internal.cc:458] Time spent opening tablet: real 0.854s user 0.010s sys 0.000s
W20250904 13:57:36.989749 12255 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:36.998342 12255 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.044935 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.074753 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.077126 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.084128 12255 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.116314 12255 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.116406 12254 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.168979 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.202108 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.205389 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.208376 12255 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.242776 12255 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.250357 12255 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.295190 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.337218 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.340595 12255 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.347838 12255 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.381460 12255 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.394397 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.434485 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.480571 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.481066 12255 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.493654 12255 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.527583 12255 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.533226 12463 meta_cache.cc:302] tablet eb9144f161da423f9fe899f96f54081c: replica 43cfdecc88a74213b3c25129a27e9a2c (127.10.98.131:40633) has failed: Network error: Client connection negotiation failed: client connection to 127.10.98.131:40633: connect: Connection refused (error 111)
W20250904 13:57:37.545893 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.582268 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.629988 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.633265 12255 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.648631 12255 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.682370 12255 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37730: Illegal state: replica 464eb7dc213646849defa0776cfedb52 is not leader of this config: current role FOLLOWER
W20250904 13:57:37.702625 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
I20250904 13:57:37.724224 12513 raft_consensus.cc:491] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:37.724695 12513 raft_consensus.cc:513] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724a4b4bc45048caa6e55b4db69ef398" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46811 } } peers { permanent_uuid: "43cfdecc88a74213b3c25129a27e9a2c" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40633 } } peers { permanent_uuid: "464eb7dc213646849defa0776cfedb52" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36205 } }
I20250904 13:57:37.726368 12513 leader_election.cc:290] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 724a4b4bc45048caa6e55b4db69ef398 (127.10.98.130:46811), 43cfdecc88a74213b3c25129a27e9a2c (127.10.98.131:40633)
I20250904 13:57:37.727532 12351 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "eb9144f161da423f9fe899f96f54081c" candidate_uuid: "464eb7dc213646849defa0776cfedb52" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "724a4b4bc45048caa6e55b4db69ef398" is_pre_election: true
I20250904 13:57:37.728523 12351 raft_consensus.cc:2466] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 464eb7dc213646849defa0776cfedb52 in term 3.
I20250904 13:57:37.729677 12242 leader_election.cc:304] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 464eb7dc213646849defa0776cfedb52, 724a4b4bc45048caa6e55b4db69ef398; no voters:
I20250904 13:57:37.730370 12513 raft_consensus.cc:2802] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20250904 13:57:37.730657 12513 raft_consensus.cc:491] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:57:37.730916 12513 raft_consensus.cc:3058] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 3 FOLLOWER]: Advancing to term 4
W20250904 13:57:37.733489 12242 leader_election.cc:336] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 43cfdecc88a74213b3c25129a27e9a2c (127.10.98.131:40633): Network error: Client connection negotiation failed: client connection to 127.10.98.131:40633: connect: Connection refused (error 111)
I20250904 13:57:37.737586 12513 raft_consensus.cc:513] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724a4b4bc45048caa6e55b4db69ef398" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46811 } } peers { permanent_uuid: "43cfdecc88a74213b3c25129a27e9a2c" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40633 } } peers { permanent_uuid: "464eb7dc213646849defa0776cfedb52" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36205 } }
I20250904 13:57:37.739646 12513 leader_election.cc:290] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [CANDIDATE]: Term 4 election: Requested vote from peers 724a4b4bc45048caa6e55b4db69ef398 (127.10.98.130:46811), 43cfdecc88a74213b3c25129a27e9a2c (127.10.98.131:40633)
I20250904 13:57:37.740425 12351 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "eb9144f161da423f9fe899f96f54081c" candidate_uuid: "464eb7dc213646849defa0776cfedb52" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "724a4b4bc45048caa6e55b4db69ef398"
I20250904 13:57:37.740907 12351 raft_consensus.cc:3058] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 3 FOLLOWER]: Advancing to term 4
W20250904 13:57:37.743175 12242 leader_election.cc:336] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 43cfdecc88a74213b3c25129a27e9a2c (127.10.98.131:40633): Network error: Client connection negotiation failed: client connection to 127.10.98.131:40633: connect: Connection refused (error 111)
W20250904 13:57:37.743737 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
I20250904 13:57:37.746734 12351 raft_consensus.cc:2466] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 464eb7dc213646849defa0776cfedb52 in term 4.
I20250904 13:57:37.747624 12242 leader_election.cc:304] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 464eb7dc213646849defa0776cfedb52, 724a4b4bc45048caa6e55b4db69ef398; no voters: 43cfdecc88a74213b3c25129a27e9a2c
I20250904 13:57:37.748251 12513 raft_consensus.cc:2802] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 4 FOLLOWER]: Leader election won for term 4
I20250904 13:57:37.749120 12513 raft_consensus.cc:695] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 4 LEADER]: Becoming Leader. State: Replica: 464eb7dc213646849defa0776cfedb52, State: Running, Role: LEADER
I20250904 13:57:37.749944 12513 consensus_queue.cc:237] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724a4b4bc45048caa6e55b4db69ef398" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46811 } } peers { permanent_uuid: "43cfdecc88a74213b3c25129a27e9a2c" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40633 } } peers { permanent_uuid: "464eb7dc213646849defa0776cfedb52" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36205 } }
I20250904 13:57:37.757324 12171 catalog_manager.cc:5582] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 reported cstate change: term changed from 1 to 4, leader changed from 43cfdecc88a74213b3c25129a27e9a2c (127.10.98.131) to 464eb7dc213646849defa0776cfedb52 (127.10.98.129). New cstate: current_term: 4 leader_uuid: "464eb7dc213646849defa0776cfedb52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "724a4b4bc45048caa6e55b4db69ef398" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "43cfdecc88a74213b3c25129a27e9a2c" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "464eb7dc213646849defa0776cfedb52" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36205 } health_report { overall_health: HEALTHY } } }
W20250904 13:57:37.794615 12331 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:55212: Illegal state: replica 724a4b4bc45048caa6e55b4db69ef398 is not leader of this config: current role FOLLOWER
I20250904 13:57:37.806154 12351 raft_consensus.cc:1273] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 4 FOLLOWER]: Refusing update from remote peer 464eb7dc213646849defa0776cfedb52: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20250904 13:57:37.807574 12516 consensus_queue.cc:1035] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "724a4b4bc45048caa6e55b4db69ef398" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 46811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20250904 13:57:37.808269 12242 consensus_peers.cc:489] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 -> Peer 43cfdecc88a74213b3c25129a27e9a2c (127.10.98.131:40633): Couldn't send request to peer 43cfdecc88a74213b3c25129a27e9a2c. Status: Network error: Client connection negotiation failed: client connection to 127.10.98.131:40633: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 13:57:37.875335 10634 tablet_server.cc:178] TabletServer@127.10.98.129:0 shutting down...
I20250904 13:57:37.898993 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:57:37.899606 10634 tablet_replica.cc:331] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52: stopping tablet replica
I20250904 13:57:37.900115 10634 raft_consensus.cc:2241] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 4 LEADER]: Raft consensus shutting down.
I20250904 13:57:37.901252 10634 raft_consensus.cc:2270] T eb9144f161da423f9fe899f96f54081c P 464eb7dc213646849defa0776cfedb52 [term 4 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:37.922386 10634 tablet_server.cc:195] TabletServer@127.10.98.129:0 shutdown complete.
I20250904 13:57:37.932268 10634 tablet_server.cc:178] TabletServer@127.10.98.130:0 shutting down...
I20250904 13:57:37.948570 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:57:37.949198 10634 tablet_replica.cc:331] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398: stopping tablet replica
I20250904 13:57:37.949744 10634 raft_consensus.cc:2241] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 4 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:37.950446 10634 raft_consensus.cc:2270] T eb9144f161da423f9fe899f96f54081c P 724a4b4bc45048caa6e55b4db69ef398 [term 4 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:37.961354 10634 tablet_server.cc:195] TabletServer@127.10.98.130:0 shutdown complete.
I20250904 13:57:37.971299 10634 master.cc:561] Master@127.10.98.190:46161 shutting down...
I20250904 13:57:37.987908 10634 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:57:37.988474 10634 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:37.988802 10634 tablet_replica.cc:331] T 00000000000000000000000000000000 P 582ef7e148054118aa4140bd2bd04faa: stopping tablet replica
I20250904 13:57:38.007242 10634 master.cc:583] Master@127.10.98.190:46161 shutdown complete.
[ OK ] TServerQuiescingITest.TestDoesntAllowNewScansLeadersOnly (3206 ms)
[ RUN ] TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp
I20250904 13:57:38.031159 10634 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.98.190:34251
I20250904 13:57:38.032377 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:38.037302 12530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:38.037472 12531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:38.039111 12533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:38.039923 10634 server_base.cc:1047] running on GCE node
I20250904 13:57:38.040834 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:38.041095 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:38.041289 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994258041268 us; error 0 us; skew 500 ppm
I20250904 13:57:38.041980 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:38.045142 10634 webserver.cc:480] Webserver started at http://127.10.98.190:34259/ using document root <none> and password file <none>
I20250904 13:57:38.045819 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:38.046054 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:38.046425 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:38.047936 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/master-0-root/instance:
uuid: "401ebe8008d74201aa10690b5e277332"
format_stamp: "Formatted at 2025-09-04 13:57:38 on dist-test-slave-jx76"
I20250904 13:57:38.054039 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20250904 13:57:38.058993 12538 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:38.059844 10634 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20250904 13:57:38.060123 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/master-0-root
uuid: "401ebe8008d74201aa10690b5e277332"
format_stamp: "Formatted at 2025-09-04 13:57:38 on dist-test-slave-jx76"
I20250904 13:57:38.060407 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:38.074113 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:38.075291 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:38.112049 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.190:34251
I20250904 13:57:38.112134 12589 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.190:34251 every 8 connection(s)
I20250904 13:57:38.116045 12590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:38.126978 12590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332: Bootstrap starting.
I20250904 13:57:38.131453 12590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:38.135881 12590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332: No bootstrap required, opened a new log
I20250904 13:57:38.137871 12590 raft_consensus.cc:357] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "401ebe8008d74201aa10690b5e277332" member_type: VOTER }
I20250904 13:57:38.138347 12590 raft_consensus.cc:383] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:38.138584 12590 raft_consensus.cc:738] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 401ebe8008d74201aa10690b5e277332, State: Initialized, Role: FOLLOWER
I20250904 13:57:38.139173 12590 consensus_queue.cc:260] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "401ebe8008d74201aa10690b5e277332" member_type: VOTER }
I20250904 13:57:38.139595 12590 raft_consensus.cc:397] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:57:38.139828 12590 raft_consensus.cc:491] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250904 13:57:38.140105 12590 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:38.144994 12590 raft_consensus.cc:513] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "401ebe8008d74201aa10690b5e277332" member_type: VOTER }
I20250904 13:57:38.145555 12590 leader_election.cc:304] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 401ebe8008d74201aa10690b5e277332; no voters:
I20250904 13:57:38.146888 12590 leader_election.cc:290] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:57:38.147199 12593 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:38.148593 12593 raft_consensus.cc:695] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332 [term 1 LEADER]: Becoming Leader. State: Replica: 401ebe8008d74201aa10690b5e277332, State: Running, Role: LEADER
I20250904 13:57:38.149307 12593 consensus_queue.cc:237] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "401ebe8008d74201aa10690b5e277332" member_type: VOTER }
I20250904 13:57:38.149926 12590 sys_catalog.cc:564] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 13:57:38.153242 12595 sys_catalog.cc:455] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 401ebe8008d74201aa10690b5e277332. Latest consensus state: current_term: 1 leader_uuid: "401ebe8008d74201aa10690b5e277332" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "401ebe8008d74201aa10690b5e277332" member_type: VOTER } }
I20250904 13:57:38.153095 12594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "401ebe8008d74201aa10690b5e277332" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "401ebe8008d74201aa10690b5e277332" member_type: VOTER } }
I20250904 13:57:38.154300 12595 sys_catalog.cc:458] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332 [sys.catalog]: This master's current role is: LEADER
I20250904 13:57:38.154670 12594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332 [sys.catalog]: This master's current role is: LEADER
I20250904 13:57:38.160214 12600 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 13:57:38.168190 12600 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 13:57:38.169512 10634 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 13:57:38.177659 12600 catalog_manager.cc:1349] Generated new cluster ID: 00f32325106749cbbe4cd78cc45ee054
I20250904 13:57:38.178018 12600 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 13:57:38.220592 12600 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 13:57:38.221968 12600 catalog_manager.cc:1506] Loading token signing keys...
I20250904 13:57:38.236588 12600 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332: Generated new TSK 0
I20250904 13:57:38.237293 12600 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 13:57:38.301301 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:38.307355 12611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:38.308420 12612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:38.311151 10634 server_base.cc:1047] running on GCE node
W20250904 13:57:38.311079 12614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:38.312139 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:38.312341 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:38.312505 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994258312488 us; error 0 us; skew 500 ppm
I20250904 13:57:38.313004 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:38.315398 10634 webserver.cc:480] Webserver started at http://127.10.98.129:44155/ using document root <none> and password file <none>
I20250904 13:57:38.315909 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:38.316097 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:38.316346 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:38.317473 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/ts-0-root/instance:
uuid: "4e9180d14b544e359a8b7941fc638c2e"
format_stamp: "Formatted at 2025-09-04 13:57:38 on dist-test-slave-jx76"
I20250904 13:57:38.322098 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20250904 13:57:38.325616 12619 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:38.326413 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20250904 13:57:38.326722 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/ts-0-root
uuid: "4e9180d14b544e359a8b7941fc638c2e"
format_stamp: "Formatted at 2025-09-04 13:57:38 on dist-test-slave-jx76"
I20250904 13:57:38.327026 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:38.344985 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:38.346086 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:38.347635 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:57:38.349987 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:57:38.350216 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20250904 13:57:38.350436 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:57:38.350591 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:38.389885 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.129:46225
I20250904 13:57:38.390003 12681 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.129:46225 every 8 connection(s)
I20250904 13:57:38.394698 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:38.402343 12686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:38.402587 12687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:38.405469 12689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:38.406652 10634 server_base.cc:1047] running on GCE node
I20250904 13:57:38.407486 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:38.407687 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:38.407826 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994258407806 us; error 0 us; skew 500 ppm
I20250904 13:57:38.408246 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:38.411129 10634 webserver.cc:480] Webserver started at http://127.10.98.130:35227/ using document root <none> and password file <none>
I20250904 13:57:38.411789 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:38.412027 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:38.412303 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:38.412613 12682 heartbeater.cc:344] Connected to a master server at 127.10.98.190:34251
I20250904 13:57:38.412986 12682 heartbeater.cc:461] Registering TS with master...
I20250904 13:57:38.413542 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/ts-1-root/instance:
uuid: "03f4f844e6f04885800c343d704fadb3"
format_stamp: "Formatted at 2025-09-04 13:57:38 on dist-test-slave-jx76"
I20250904 13:57:38.413717 12682 heartbeater.cc:507] Master 127.10.98.190:34251 requested a full tablet report, sending...
I20250904 13:57:38.416145 12555 ts_manager.cc:194] Registered new tserver with Master: 4e9180d14b544e359a8b7941fc638c2e (127.10.98.129:46225)
I20250904 13:57:38.418382 12555 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37430
I20250904 13:57:38.418692 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20250904 13:57:38.422345 12694 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:38.423128 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250904 13:57:38.423434 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/ts-1-root
uuid: "03f4f844e6f04885800c343d704fadb3"
format_stamp: "Formatted at 2025-09-04 13:57:38 on dist-test-slave-jx76"
I20250904 13:57:38.423730 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:38.442974 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:38.444123 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:38.445683 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:57:38.448210 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:57:38.448431 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:38.448664 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:57:38.448825 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:38.490993 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.130:41181
I20250904 13:57:38.491107 12756 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.130:41181 every 8 connection(s)
I20250904 13:57:38.495829 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:38.502602 12760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:38.503531 12761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:38.508626 12757 heartbeater.cc:344] Connected to a master server at 127.10.98.190:34251
I20250904 13:57:38.509037 12757 heartbeater.cc:461] Registering TS with master...
I20250904 13:57:38.509773 12757 heartbeater.cc:507] Master 127.10.98.190:34251 requested a full tablet report, sending...
I20250904 13:57:38.510686 10634 server_base.cc:1047] running on GCE node
I20250904 13:57:38.511931 12555 ts_manager.cc:194] Registered new tserver with Master: 03f4f844e6f04885800c343d704fadb3 (127.10.98.130:41181)
W20250904 13:57:38.512516 12763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:38.513574 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20250904 13:57:38.513756 12555 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37432
W20250904 13:57:38.513880 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:38.514231 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994258514208 us; error 0 us; skew 500 ppm
I20250904 13:57:38.514995 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:38.517550 10634 webserver.cc:480] Webserver started at http://127.10.98.131:43317/ using document root <none> and password file <none>
I20250904 13:57:38.518064 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:38.518312 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:38.518574 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:38.519690 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/ts-2-root/instance:
uuid: "2e70be752a5b4b338664b8dca60664fc"
format_stamp: "Formatted at 2025-09-04 13:57:38 on dist-test-slave-jx76"
I20250904 13:57:38.524296 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250904 13:57:38.527861 12768 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:38.528641 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20250904 13:57:38.528970 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/ts-2-root
uuid: "2e70be752a5b4b338664b8dca60664fc"
format_stamp: "Formatted at 2025-09-04 13:57:38 on dist-test-slave-jx76"
I20250904 13:57:38.529304 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:38.541326 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:38.542508 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:38.543967 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:57:38.546331 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:57:38.546546 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:38.546778 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:57:38.546931 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:38.586614 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.131:42717
I20250904 13:57:38.586716 12830 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.131:42717 every 8 connection(s)
I20250904 13:57:38.599910 12831 heartbeater.cc:344] Connected to a master server at 127.10.98.190:34251
I20250904 13:57:38.600297 12831 heartbeater.cc:461] Registering TS with master...
I20250904 13:57:38.600996 12831 heartbeater.cc:507] Master 127.10.98.190:34251 requested a full tablet report, sending...
I20250904 13:57:38.602938 12555 ts_manager.cc:194] Registered new tserver with Master: 2e70be752a5b4b338664b8dca60664fc (127.10.98.131:42717)
I20250904 13:57:38.603482 10634 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01368995s
I20250904 13:57:38.604791 12555 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37436
I20250904 13:57:38.639952 10634 test_util.cc:276] Using random seed: 447257655
I20250904 13:57:38.662525 12555 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37466:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 13:57:38.664412 12555 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 13:57:38.705008 12722 tablet_service.cc:1468] Processing CreateTablet for tablet 2c937394b8c043b8bc46b39a45ebf2b4 (DEFAULT_TABLE table=test-workload [id=0361d9f8ff6748399e089964010fd752]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:57:38.706444 12722 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c937394b8c043b8bc46b39a45ebf2b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:38.706393 12647 tablet_service.cc:1468] Processing CreateTablet for tablet 2c937394b8c043b8bc46b39a45ebf2b4 (DEFAULT_TABLE table=test-workload [id=0361d9f8ff6748399e089964010fd752]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:57:38.706547 12796 tablet_service.cc:1468] Processing CreateTablet for tablet 2c937394b8c043b8bc46b39a45ebf2b4 (DEFAULT_TABLE table=test-workload [id=0361d9f8ff6748399e089964010fd752]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:57:38.707857 12647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c937394b8c043b8bc46b39a45ebf2b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:38.707857 12796 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c937394b8c043b8bc46b39a45ebf2b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:38.723580 12852 tablet_bootstrap.cc:492] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: Bootstrap starting.
I20250904 13:57:38.726567 12853 tablet_bootstrap.cc:492] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e: Bootstrap starting.
I20250904 13:57:38.727392 12854 tablet_bootstrap.cc:492] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: Bootstrap starting.
I20250904 13:57:38.732381 12852 tablet_bootstrap.cc:654] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:38.732889 12854 tablet_bootstrap.cc:654] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:38.733718 12853 tablet_bootstrap.cc:654] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:38.737596 12854 tablet_bootstrap.cc:492] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: No bootstrap required, opened a new log
I20250904 13:57:38.738297 12854 ts_tablet_manager.cc:1397] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.008s
I20250904 13:57:38.739725 12853 tablet_bootstrap.cc:492] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e: No bootstrap required, opened a new log
I20250904 13:57:38.740092 12852 tablet_bootstrap.cc:492] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: No bootstrap required, opened a new log
I20250904 13:57:38.740658 12852 ts_tablet_manager.cc:1397] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.008s
I20250904 13:57:38.740929 12854 raft_consensus.cc:357] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e9180d14b544e359a8b7941fc638c2e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 46225 } } peers { permanent_uuid: "03f4f844e6f04885800c343d704fadb3" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41181 } } peers { permanent_uuid: "2e70be752a5b4b338664b8dca60664fc" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 42717 } }
I20250904 13:57:38.741865 12854 raft_consensus.cc:383] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:38.741544 12853 ts_tablet_manager.cc:1397] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.007s
I20250904 13:57:38.742277 12854 raft_consensus.cc:738] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e70be752a5b4b338664b8dca60664fc, State: Initialized, Role: FOLLOWER
I20250904 13:57:38.743086 12854 consensus_queue.cc:260] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e9180d14b544e359a8b7941fc638c2e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 46225 } } peers { permanent_uuid: "03f4f844e6f04885800c343d704fadb3" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41181 } } peers { permanent_uuid: "2e70be752a5b4b338664b8dca60664fc" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 42717 } }
I20250904 13:57:38.743647 12852 raft_consensus.cc:357] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e9180d14b544e359a8b7941fc638c2e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 46225 } } peers { permanent_uuid: "03f4f844e6f04885800c343d704fadb3" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41181 } } peers { permanent_uuid: "2e70be752a5b4b338664b8dca60664fc" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 42717 } }
I20250904 13:57:38.744426 12852 raft_consensus.cc:383] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:38.744347 12853 raft_consensus.cc:357] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e9180d14b544e359a8b7941fc638c2e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 46225 } } peers { permanent_uuid: "03f4f844e6f04885800c343d704fadb3" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41181 } } peers { permanent_uuid: "2e70be752a5b4b338664b8dca60664fc" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 42717 } }
I20250904 13:57:38.744964 12852 raft_consensus.cc:738] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03f4f844e6f04885800c343d704fadb3, State: Initialized, Role: FOLLOWER
I20250904 13:57:38.745277 12853 raft_consensus.cc:383] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:38.745533 12831 heartbeater.cc:499] Master 127.10.98.190:34251 was elected leader, sending a full tablet report...
I20250904 13:57:38.745661 12853 raft_consensus.cc:738] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e9180d14b544e359a8b7941fc638c2e, State: Initialized, Role: FOLLOWER
I20250904 13:57:38.746271 12854 ts_tablet_manager.cc:1428] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: Time spent starting tablet: real 0.008s user 0.003s sys 0.005s
I20250904 13:57:38.745944 12852 consensus_queue.cc:260] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e9180d14b544e359a8b7941fc638c2e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 46225 } } peers { permanent_uuid: "03f4f844e6f04885800c343d704fadb3" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41181 } } peers { permanent_uuid: "2e70be752a5b4b338664b8dca60664fc" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 42717 } }
I20250904 13:57:38.746515 12853 consensus_queue.cc:260] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e9180d14b544e359a8b7941fc638c2e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 46225 } } peers { permanent_uuid: "03f4f844e6f04885800c343d704fadb3" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41181 } } peers { permanent_uuid: "2e70be752a5b4b338664b8dca60664fc" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 42717 } }
I20250904 13:57:38.749238 12682 heartbeater.cc:499] Master 127.10.98.190:34251 was elected leader, sending a full tablet report...
I20250904 13:57:38.751681 12853 ts_tablet_manager.cc:1428] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e: Time spent starting tablet: real 0.009s user 0.000s sys 0.007s
I20250904 13:57:38.753522 12757 heartbeater.cc:499] Master 127.10.98.190:34251 was elected leader, sending a full tablet report...
I20250904 13:57:38.754190 12852 ts_tablet_manager.cc:1428] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: Time spent starting tablet: real 0.013s user 0.010s sys 0.003s
I20250904 13:57:38.782052 12860 raft_consensus.cc:491] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:57:38.782622 12860 raft_consensus.cc:513] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e9180d14b544e359a8b7941fc638c2e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 46225 } } peers { permanent_uuid: "03f4f844e6f04885800c343d704fadb3" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41181 } } peers { permanent_uuid: "2e70be752a5b4b338664b8dca60664fc" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 42717 } }
I20250904 13:57:38.784859 12860 leader_election.cc:290] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03f4f844e6f04885800c343d704fadb3 (127.10.98.130:41181), 2e70be752a5b4b338664b8dca60664fc (127.10.98.131:42717)
I20250904 13:57:38.795303 12732 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2c937394b8c043b8bc46b39a45ebf2b4" candidate_uuid: "4e9180d14b544e359a8b7941fc638c2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03f4f844e6f04885800c343d704fadb3" is_pre_election: true
I20250904 13:57:38.796128 12732 raft_consensus.cc:2466] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e9180d14b544e359a8b7941fc638c2e in term 0.
I20250904 13:57:38.797045 12806 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2c937394b8c043b8bc46b39a45ebf2b4" candidate_uuid: "4e9180d14b544e359a8b7941fc638c2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e70be752a5b4b338664b8dca60664fc" is_pre_election: true
I20250904 13:57:38.797282 12620 leader_election.cc:304] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03f4f844e6f04885800c343d704fadb3, 4e9180d14b544e359a8b7941fc638c2e; no voters:
I20250904 13:57:38.797657 12806 raft_consensus.cc:2466] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e9180d14b544e359a8b7941fc638c2e in term 0.
I20250904 13:57:38.798063 12860 raft_consensus.cc:2802] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:57:38.798473 12860 raft_consensus.cc:491] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:57:38.798750 12860 raft_consensus.cc:3058] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:38.803972 12860 raft_consensus.cc:513] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e9180d14b544e359a8b7941fc638c2e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 46225 } } peers { permanent_uuid: "03f4f844e6f04885800c343d704fadb3" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41181 } } peers { permanent_uuid: "2e70be752a5b4b338664b8dca60664fc" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 42717 } }
I20250904 13:57:38.805615 12860 leader_election.cc:290] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [CANDIDATE]: Term 1 election: Requested vote from peers 03f4f844e6f04885800c343d704fadb3 (127.10.98.130:41181), 2e70be752a5b4b338664b8dca60664fc (127.10.98.131:42717)
I20250904 13:57:38.806703 12732 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2c937394b8c043b8bc46b39a45ebf2b4" candidate_uuid: "4e9180d14b544e359a8b7941fc638c2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03f4f844e6f04885800c343d704fadb3"
I20250904 13:57:38.806703 12806 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2c937394b8c043b8bc46b39a45ebf2b4" candidate_uuid: "4e9180d14b544e359a8b7941fc638c2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e70be752a5b4b338664b8dca60664fc"
I20250904 13:57:38.807278 12732 raft_consensus.cc:3058] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:38.807354 12806 raft_consensus.cc:3058] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:38.813764 12732 raft_consensus.cc:2466] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e9180d14b544e359a8b7941fc638c2e in term 1.
I20250904 13:57:38.813973 12806 raft_consensus.cc:2466] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e9180d14b544e359a8b7941fc638c2e in term 1.
I20250904 13:57:38.814987 12620 leader_election.cc:304] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03f4f844e6f04885800c343d704fadb3, 4e9180d14b544e359a8b7941fc638c2e; no voters:
I20250904 13:57:38.815788 12860 raft_consensus.cc:2802] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:38.816913 12860 raft_consensus.cc:695] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [term 1 LEADER]: Becoming Leader. State: Replica: 4e9180d14b544e359a8b7941fc638c2e, State: Running, Role: LEADER
I20250904 13:57:38.817593 12860 consensus_queue.cc:237] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e9180d14b544e359a8b7941fc638c2e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 46225 } } peers { permanent_uuid: "03f4f844e6f04885800c343d704fadb3" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41181 } } peers { permanent_uuid: "2e70be752a5b4b338664b8dca60664fc" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 42717 } }
I20250904 13:57:38.826301 12555 catalog_manager.cc:5582] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e9180d14b544e359a8b7941fc638c2e (127.10.98.129). New cstate: current_term: 1 leader_uuid: "4e9180d14b544e359a8b7941fc638c2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e9180d14b544e359a8b7941fc638c2e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 46225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "03f4f844e6f04885800c343d704fadb3" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e70be752a5b4b338664b8dca60664fc" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 42717 } health_report { overall_health: UNKNOWN } } }
I20250904 13:57:38.946780 12732 raft_consensus.cc:1273] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3 [term 1 FOLLOWER]: Refusing update from remote peer 4e9180d14b544e359a8b7941fc638c2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 13:57:38.946780 12806 raft_consensus.cc:1273] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [term 1 FOLLOWER]: Refusing update from remote peer 4e9180d14b544e359a8b7941fc638c2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 13:57:38.948470 12862 consensus_queue.cc:1035] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "03f4f844e6f04885800c343d704fadb3" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:57:38.949080 12860 consensus_queue.cc:1035] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e70be752a5b4b338664b8dca60664fc" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 42717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:57:45.899472 12722 tablet_service.cc:1515] Processing DeleteTablet for tablet 2c937394b8c043b8bc46b39a45ebf2b4 with delete_type TABLET_DATA_TOMBSTONED from {username='slave'} at 127.0.0.1:52542
I20250904 13:57:45.901311 12905 tablet_replica.cc:331] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: stopping tablet replica
I20250904 13:57:45.902043 12905 raft_consensus.cc:2241] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:45.902813 12905 raft_consensus.cc:2270] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:45.924378 12905 ts_tablet_manager.cc:1905] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250904 13:57:45.937729 12905 ts_tablet_manager.cc:1918] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.207
I20250904 13:57:45.938270 12905 log.cc:1199] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: Deleting WAL directory at /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/ts-1-root/wals/2c937394b8c043b8bc46b39a45ebf2b4
I20250904 13:57:45.953163 12796 tablet_service.cc:1515] Processing DeleteTablet for tablet 2c937394b8c043b8bc46b39a45ebf2b4 with delete_type TABLET_DATA_TOMBSTONED from {username='slave'} at 127.0.0.1:57814
I20250904 13:57:45.954702 12907 tablet_replica.cc:331] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: stopping tablet replica
I20250904 13:57:45.955374 12907 raft_consensus.cc:2241] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:45.956053 12907 raft_consensus.cc:2270] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:45.977823 12907 ts_tablet_manager.cc:1905] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20250904 13:57:45.989879 12907 ts_tablet_manager.cc:1918] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.207
I20250904 13:57:45.990307 12907 log.cc:1199] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: Deleting WAL directory at /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp.1756994241738102-10634-0/minicluster-data/ts-2-root/wals/2c937394b8c043b8bc46b39a45ebf2b4
W20250904 13:57:46.043157 12620 consensus_peers.cc:489] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e -> Peer 03f4f844e6f04885800c343d704fadb3 (127.10.98.130:41181): Couldn't send request to peer 03f4f844e6f04885800c343d704fadb3. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
W20250904 13:57:46.132651 12623 consensus_peers.cc:489] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e -> Peer 2e70be752a5b4b338664b8dca60664fc (127.10.98.131:42717): Couldn't send request to peer 2e70be752a5b4b338664b8dca60664fc. Error code: TABLET_NOT_FOUND (6). Status: Illegal state: Tablet not RUNNING: STOPPED. This is attempt 1: this message will repeat every 5th retry.
I20250904 13:57:46.281996 12910 ts_tablet_manager.cc:927] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: Initiating tablet copy from peer 4e9180d14b544e359a8b7941fc638c2e (127.10.98.129:46225)
I20250904 13:57:46.285249 12910 tablet_copy_client.cc:323] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: tablet copy: Beginning tablet copy session from remote peer at address 127.10.98.129:46225
I20250904 13:57:46.296949 12667 tablet_copy_service.cc:140] P 4e9180d14b544e359a8b7941fc638c2e: Received BeginTabletCopySession request for tablet 2c937394b8c043b8bc46b39a45ebf2b4 from peer 03f4f844e6f04885800c343d704fadb3 ({username='slave'} at 127.0.0.1:58514)
I20250904 13:57:46.297726 12667 tablet_copy_service.cc:161] P 4e9180d14b544e359a8b7941fc638c2e: Beginning new tablet copy session on tablet 2c937394b8c043b8bc46b39a45ebf2b4 from peer 03f4f844e6f04885800c343d704fadb3 at {username='slave'} at 127.0.0.1:58514: session id = 03f4f844e6f04885800c343d704fadb3-2c937394b8c043b8bc46b39a45ebf2b4
I20250904 13:57:46.305456 12667 tablet_copy_source_session.cc:215] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e: Tablet Copy: opened 0 blocks and 1 log segments
I20250904 13:57:46.308300 12910 ts_tablet_manager.cc:1905] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: Deleting tablet data with delete state TABLET_DATA_COPYING
I20250904 13:57:46.320264 12910 ts_tablet_manager.cc:1918] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.207
I20250904 13:57:46.321026 12910 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c937394b8c043b8bc46b39a45ebf2b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:46.326948 12910 tablet_copy_client.cc:806] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: tablet copy: Starting download of 0 data blocks...
I20250904 13:57:46.327550 12910 tablet_copy_client.cc:670] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: tablet copy: Starting download of 1 WAL segments...
I20250904 13:57:46.331118 12911 tablet_copy_client.cc:1025] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: tablet copy: Injecting latency into file download: 500
I20250904 13:57:46.489645 12914 ts_tablet_manager.cc:927] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: Initiating tablet copy from peer 4e9180d14b544e359a8b7941fc638c2e (127.10.98.129:46225)
I20250904 13:57:46.491328 12914 tablet_copy_client.cc:323] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: tablet copy: Beginning tablet copy session from remote peer at address 127.10.98.129:46225
I20250904 13:57:46.501272 12667 tablet_copy_service.cc:140] P 4e9180d14b544e359a8b7941fc638c2e: Received BeginTabletCopySession request for tablet 2c937394b8c043b8bc46b39a45ebf2b4 from peer 2e70be752a5b4b338664b8dca60664fc ({username='slave'} at 127.0.0.1:58518)
I20250904 13:57:46.501780 12667 tablet_copy_service.cc:161] P 4e9180d14b544e359a8b7941fc638c2e: Beginning new tablet copy session on tablet 2c937394b8c043b8bc46b39a45ebf2b4 from peer 2e70be752a5b4b338664b8dca60664fc at {username='slave'} at 127.0.0.1:58518: session id = 2e70be752a5b4b338664b8dca60664fc-2c937394b8c043b8bc46b39a45ebf2b4
I20250904 13:57:46.507895 12667 tablet_copy_source_session.cc:215] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e: Tablet Copy: opened 0 blocks and 1 log segments
I20250904 13:57:46.510744 12914 ts_tablet_manager.cc:1905] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: Deleting tablet data with delete state TABLET_DATA_COPYING
I20250904 13:57:46.523072 12914 ts_tablet_manager.cc:1918] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: tablet deleted with delete type TABLET_DATA_COPYING: last-logged OpId 1.207
I20250904 13:57:46.523756 12914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c937394b8c043b8bc46b39a45ebf2b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:46.529894 12914 tablet_copy_client.cc:806] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: tablet copy: Starting download of 0 data blocks...
I20250904 13:57:46.530431 12914 tablet_copy_client.cc:670] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: tablet copy: Starting download of 1 WAL segments...
I20250904 13:57:46.533342 12915 tablet_copy_client.cc:1025] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: tablet copy: Injecting latency into file download: 500
I20250904 13:57:46.834059 12911 tablet_copy_client.cc:1025] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: tablet copy: Injecting latency into file download: 500
I20250904 13:57:47.035907 12915 tablet_copy_client.cc:1025] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: tablet copy: Injecting latency into file download: 500
I20250904 13:57:47.337325 12911 tablet_copy_client.cc:1025] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: tablet copy: Injecting latency into file download: 500
I20250904 13:57:47.538643 12915 tablet_copy_client.cc:1025] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: tablet copy: Injecting latency into file download: 500
I20250904 13:57:47.840508 12911 tablet_copy_client.cc:1025] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: tablet copy: Injecting latency into file download: 500
W20250904 13:57:47.861513 12623 consensus_peers.cc:489] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e -> Peer 2e70be752a5b4b338664b8dca60664fc (127.10.98.131:42717): Couldn't send request to peer 2e70be752a5b4b338664b8dca60664fc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20250904 13:57:47.876250 12620 consensus_peers.cc:489] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e -> Peer 03f4f844e6f04885800c343d704fadb3 (127.10.98.130:41181): Couldn't send request to peer 03f4f844e6f04885800c343d704fadb3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20250904 13:57:48.041529 12915 tablet_copy_client.cc:1025] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: tablet copy: Injecting latency into file download: 500
I20250904 13:57:48.343905 12911 tablet_copy_client.cc:1025] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: tablet copy: Injecting latency into file download: 500
I20250904 13:57:48.544548 12915 tablet_copy_client.cc:1025] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: tablet copy: Injecting latency into file download: 500
I20250904 13:57:48.847220 12911 tablet_copy_client.cc:1025] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: tablet copy: Injecting latency into file download: 500
W20250904 13:57:49.264848 12620 consensus_peers.cc:489] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e -> Peer 03f4f844e6f04885800c343d704fadb3 (127.10.98.130:41181): Couldn't send request to peer 03f4f844e6f04885800c343d704fadb3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20250904 13:57:49.402688 12623 consensus_peers.cc:489] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e -> Peer 2e70be752a5b4b338664b8dca60664fc (127.10.98.131:42717): Couldn't send request to peer 2e70be752a5b4b338664b8dca60664fc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20250904 13:57:50.300489 12914 tablet_copy_client.cc:538] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250904 13:57:50.311004 12914 tablet_bootstrap.cc:492] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: Bootstrap starting.
W20250904 13:57:50.645751 12620 consensus_peers.cc:489] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e -> Peer 03f4f844e6f04885800c343d704fadb3 (127.10.98.130:41181): Couldn't send request to peer 03f4f844e6f04885800c343d704fadb3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 16: this message will repeat every 5th retry.
I20250904 13:57:50.651630 12910 tablet_copy_client.cc:538] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20250904 13:57:50.661717 12910 tablet_bootstrap.cc:492] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: Bootstrap starting.
W20250904 13:57:50.794885 12623 consensus_peers.cc:489] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e -> Peer 2e70be752a5b4b338664b8dca60664fc (127.10.98.131:42717): Couldn't send request to peer 2e70be752a5b4b338664b8dca60664fc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 16: this message will repeat every 5th retry.
W20250904 13:57:52.167896 12620 consensus_peers.cc:489] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e -> Peer 03f4f844e6f04885800c343d704fadb3 (127.10.98.130:41181): Couldn't send request to peer 03f4f844e6f04885800c343d704fadb3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 21: this message will repeat every 5th retry.
W20250904 13:57:52.177433 12623 consensus_peers.cc:489] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e -> Peer 2e70be752a5b4b338664b8dca60664fc (127.10.98.131:42717): Couldn't send request to peer 2e70be752a5b4b338664b8dca60664fc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 21: this message will repeat every 5th retry.
I20250904 13:57:52.919863 12914 tablet_bootstrap.cc:492] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=207 overwritten=0 applied=207 ignored=0} inserts{seen=10300 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250904 13:57:52.921185 12914 tablet_bootstrap.cc:492] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: Bootstrap complete.
I20250904 13:57:52.922205 12914 ts_tablet_manager.cc:1397] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: Time spent bootstrapping tablet: real 2.611s user 2.531s sys 0.064s
I20250904 13:57:52.925257 12914 raft_consensus.cc:357] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e9180d14b544e359a8b7941fc638c2e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 46225 } } peers { permanent_uuid: "03f4f844e6f04885800c343d704fadb3" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41181 } } peers { permanent_uuid: "2e70be752a5b4b338664b8dca60664fc" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 42717 } }
I20250904 13:57:52.926373 12914 raft_consensus.cc:738] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e70be752a5b4b338664b8dca60664fc, State: Initialized, Role: FOLLOWER
I20250904 13:57:52.927129 12914 consensus_queue.cc:260] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 207, Last appended: 1.207, Last appended by leader: 207, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e9180d14b544e359a8b7941fc638c2e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 46225 } } peers { permanent_uuid: "03f4f844e6f04885800c343d704fadb3" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41181 } } peers { permanent_uuid: "2e70be752a5b4b338664b8dca60664fc" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 42717 } }
I20250904 13:57:52.930424 12914 ts_tablet_manager.cc:1428] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20250904 13:57:52.933279 12667 tablet_copy_service.cc:342] P 4e9180d14b544e359a8b7941fc638c2e: Request end of tablet copy session 2e70be752a5b4b338664b8dca60664fc-2c937394b8c043b8bc46b39a45ebf2b4 received from {username='slave'} at 127.0.0.1:58518
I20250904 13:57:52.933991 12667 tablet_copy_service.cc:434] P 4e9180d14b544e359a8b7941fc638c2e: ending tablet copy session 2e70be752a5b4b338664b8dca60664fc-2c937394b8c043b8bc46b39a45ebf2b4 on tablet 2c937394b8c043b8bc46b39a45ebf2b4 with peer 2e70be752a5b4b338664b8dca60664fc
I20250904 13:57:53.028493 12921 raft_consensus.cc:991] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e: : Instructing follower 2e70be752a5b4b338664b8dca60664fc to start an election
I20250904 13:57:53.029311 12908 raft_consensus.cc:1079] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [term 1 LEADER]: Signalling peer 2e70be752a5b4b338664b8dca60664fc to start an election
I20250904 13:57:53.031364 12805 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "2c937394b8c043b8bc46b39a45ebf2b4"
dest_uuid: "2e70be752a5b4b338664b8dca60664fc"
from {username='slave'} at 127.0.0.1:34456
I20250904 13:57:53.031929 12805 raft_consensus.cc:491] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250904 13:57:53.032279 12805 raft_consensus.cc:3058] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [term 1 FOLLOWER]: Advancing to term 2
I20250904 13:57:53.039528 12805 raft_consensus.cc:513] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e9180d14b544e359a8b7941fc638c2e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 46225 } } peers { permanent_uuid: "03f4f844e6f04885800c343d704fadb3" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41181 } } peers { permanent_uuid: "2e70be752a5b4b338664b8dca60664fc" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 42717 } }
I20250904 13:57:53.041785 12805 leader_election.cc:290] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [CANDIDATE]: Term 2 election: Requested vote from peers 4e9180d14b544e359a8b7941fc638c2e (127.10.98.129:46225), 03f4f844e6f04885800c343d704fadb3 (127.10.98.130:41181)
I20250904 13:57:53.043314 12657 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2c937394b8c043b8bc46b39a45ebf2b4" candidate_uuid: "2e70be752a5b4b338664b8dca60664fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 207 } } ignore_live_leader: true dest_uuid: "4e9180d14b544e359a8b7941fc638c2e"
I20250904 13:57:53.044162 12657 raft_consensus.cc:3053] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [term 1 LEADER]: Stepping down as leader of term 1
I20250904 13:57:53.044520 12657 raft_consensus.cc:738] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4e9180d14b544e359a8b7941fc638c2e, State: Running, Role: LEADER
I20250904 13:57:53.045466 12657 consensus_queue.cc:260] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 207, Committed index: 207, Last appended: 1.207, Last appended by leader: 207, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e9180d14b544e359a8b7941fc638c2e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 46225 } } peers { permanent_uuid: "03f4f844e6f04885800c343d704fadb3" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41181 } } peers { permanent_uuid: "2e70be752a5b4b338664b8dca60664fc" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 42717 } }
I20250904 13:57:53.047794 12657 raft_consensus.cc:3058] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [term 1 FOLLOWER]: Advancing to term 2
I20250904 13:57:53.056985 12732 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2c937394b8c043b8bc46b39a45ebf2b4" candidate_uuid: "2e70be752a5b4b338664b8dca60664fc" candidate_term: 2 candidate_status { last_received { term: 1 index: 207 } } ignore_live_leader: true dest_uuid: "03f4f844e6f04885800c343d704fadb3"
I20250904 13:57:53.057452 12657 raft_consensus.cc:2466] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e70be752a5b4b338664b8dca60664fc in term 2.
W20250904 13:57:53.058531 12769 leader_election.cc:343] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 03f4f844e6f04885800c343d704fadb3 (127.10.98.130:41181): Illegal state: must be running to vote when last-logged opid is not known
I20250904 13:57:53.059388 12769 leader_election.cc:304] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2e70be752a5b4b338664b8dca60664fc, 4e9180d14b544e359a8b7941fc638c2e; no voters: 03f4f844e6f04885800c343d704fadb3
I20250904 13:57:53.060885 12920 raft_consensus.cc:2802] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [term 2 FOLLOWER]: Leader election won for term 2
I20250904 13:57:53.062193 12920 raft_consensus.cc:695] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [term 2 LEADER]: Becoming Leader. State: Replica: 2e70be752a5b4b338664b8dca60664fc, State: Running, Role: LEADER
I20250904 13:57:53.063278 12920 consensus_queue.cc:237] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 207, Committed index: 207, Last appended: 1.207, Last appended by leader: 207, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e9180d14b544e359a8b7941fc638c2e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 46225 } } peers { permanent_uuid: "03f4f844e6f04885800c343d704fadb3" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41181 } } peers { permanent_uuid: "2e70be752a5b4b338664b8dca60664fc" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 42717 } }
I20250904 13:57:53.074071 12550 catalog_manager.cc:5582] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc reported cstate change: term changed from 1 to 2, leader changed from 4e9180d14b544e359a8b7941fc638c2e (127.10.98.129) to 2e70be752a5b4b338664b8dca60664fc (127.10.98.131). New cstate: current_term: 2 leader_uuid: "2e70be752a5b4b338664b8dca60664fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e9180d14b544e359a8b7941fc638c2e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 46225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03f4f844e6f04885800c343d704fadb3" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e70be752a5b4b338664b8dca60664fc" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 42717 } health_report { overall_health: HEALTHY } } }
I20250904 13:57:53.244223 12910 tablet_bootstrap.cc:492] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: Bootstrap replayed 1/1 log segments. Stats: ops{read=207 overwritten=0 applied=207 ignored=0} inserts{seen=10300 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20250904 13:57:53.245127 12910 tablet_bootstrap.cc:492] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: Bootstrap complete.
I20250904 13:57:53.245899 12910 ts_tablet_manager.cc:1397] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: Time spent bootstrapping tablet: real 2.584s user 2.513s sys 0.052s
I20250904 13:57:53.247907 12910 raft_consensus.cc:357] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e9180d14b544e359a8b7941fc638c2e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 46225 } } peers { permanent_uuid: "03f4f844e6f04885800c343d704fadb3" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41181 } } peers { permanent_uuid: "2e70be752a5b4b338664b8dca60664fc" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 42717 } }
I20250904 13:57:53.248370 12910 raft_consensus.cc:738] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03f4f844e6f04885800c343d704fadb3, State: Initialized, Role: FOLLOWER
I20250904 13:57:53.248953 12910 consensus_queue.cc:260] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 207, Last appended: 1.207, Last appended by leader: 207, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e9180d14b544e359a8b7941fc638c2e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 46225 } } peers { permanent_uuid: "03f4f844e6f04885800c343d704fadb3" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41181 } } peers { permanent_uuid: "2e70be752a5b4b338664b8dca60664fc" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 42717 } }
I20250904 13:57:53.251562 12910 ts_tablet_manager.cc:1428] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20250904 13:57:53.253480 12667 tablet_copy_service.cc:342] P 4e9180d14b544e359a8b7941fc638c2e: Request end of tablet copy session 03f4f844e6f04885800c343d704fadb3-2c937394b8c043b8bc46b39a45ebf2b4 received from {username='slave'} at 127.0.0.1:58514
I20250904 13:57:53.253780 12667 tablet_copy_service.cc:434] P 4e9180d14b544e359a8b7941fc638c2e: ending tablet copy session 03f4f844e6f04885800c343d704fadb3-2c937394b8c043b8bc46b39a45ebf2b4 on tablet 2c937394b8c043b8bc46b39a45ebf2b4 with peer 03f4f844e6f04885800c343d704fadb3
I20250904 13:57:53.421986 12657 raft_consensus.cc:1273] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [term 2 FOLLOWER]: Refusing update from remote peer 2e70be752a5b4b338664b8dca60664fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 207. Preceding OpId from leader: term: 2 index: 208. (index mismatch)
I20250904 13:57:53.423326 12920 consensus_queue.cc:1035] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e9180d14b544e359a8b7941fc638c2e" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 46225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 208, Last known committed idx: 207, Time since last communication: 0.000s
I20250904 13:57:53.433774 12732 raft_consensus.cc:3058] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3 [term 1 FOLLOWER]: Advancing to term 2
I20250904 13:57:53.441972 12732 raft_consensus.cc:1273] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3 [term 2 FOLLOWER]: Refusing update from remote peer 2e70be752a5b4b338664b8dca60664fc: Log matching property violated. Preceding OpId in replica: term: 1 index: 207. Preceding OpId from leader: term: 2 index: 208. (index mismatch)
I20250904 13:57:53.443382 12920 consensus_queue.cc:1035] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "03f4f844e6f04885800c343d704fadb3" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 208, Last known committed idx: 207, Time since last communication: 0.000s
I20250904 13:57:54.104578 10634 tablet_server.cc:178] TabletServer@127.10.98.129:0 shutting down...
I20250904 13:57:54.140476 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:57:54.141239 10634 tablet_replica.cc:331] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e: stopping tablet replica
I20250904 13:57:54.141824 10634 raft_consensus.cc:2241] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [term 2 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:54.142383 10634 raft_consensus.cc:2270] T 2c937394b8c043b8bc46b39a45ebf2b4 P 4e9180d14b544e359a8b7941fc638c2e [term 2 FOLLOWER]: Raft consensus is shut down!
W20250904 13:57:54.177693 12769 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.98.129:46225: connect: Connection refused (error 111) [suppressed 7 similar messages]
W20250904 13:57:54.180589 12769 consensus_peers.cc:489] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc -> Peer 4e9180d14b544e359a8b7941fc638c2e (127.10.98.129:46225): Couldn't send request to peer 4e9180d14b544e359a8b7941fc638c2e. Status: Network error: Client connection negotiation failed: client connection to 127.10.98.129:46225: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 13:57:54.188015 10634 tablet_server.cc:195] TabletServer@127.10.98.129:0 shutdown complete.
I20250904 13:57:54.203969 10634 tablet_server.cc:178] TabletServer@127.10.98.130:0 shutting down...
I20250904 13:57:54.228547 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:57:54.229290 10634 tablet_replica.cc:331] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3: stopping tablet replica
I20250904 13:57:54.229887 10634 raft_consensus.cc:2241] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20250904 13:57:54.230362 10634 raft_consensus.cc:2270] T 2c937394b8c043b8bc46b39a45ebf2b4 P 03f4f844e6f04885800c343d704fadb3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:54.268281 10634 tablet_server.cc:195] TabletServer@127.10.98.130:0 shutdown complete.
W20250904 13:57:54.280506 12769 consensus_peers.cc:489] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc -> Peer 03f4f844e6f04885800c343d704fadb3 (127.10.98.130:41181): Couldn't send request to peer 03f4f844e6f04885800c343d704fadb3. Status: Network error: Client connection negotiation failed: client connection to 127.10.98.130:41181: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 13:57:54.281603 10634 tablet_server.cc:178] TabletServer@127.10.98.131:0 shutting down...
I20250904 13:57:54.304322 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:57:54.305065 10634 tablet_replica.cc:331] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc: stopping tablet replica
I20250904 13:57:54.305575 10634 raft_consensus.cc:2241] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [term 2 LEADER]: Raft consensus shutting down.
I20250904 13:57:54.306460 10634 raft_consensus.cc:2270] T 2c937394b8c043b8bc46b39a45ebf2b4 P 2e70be752a5b4b338664b8dca60664fc [term 2 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:54.344676 10634 tablet_server.cc:195] TabletServer@127.10.98.131:0 shutdown complete.
I20250904 13:57:54.356858 10634 master.cc:561] Master@127.10.98.190:34251 shutting down...
I20250904 13:57:54.378520 10634 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:57:54.379127 10634 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:57:54.379475 10634 tablet_replica.cc:331] T 00000000000000000000000000000000 P 401ebe8008d74201aa10690b5e277332: stopping tablet replica
I20250904 13:57:54.399682 10634 master.cc:583] Master@127.10.98.190:34251 shutdown complete.
[ OK ] TServerQuiescingITest.TestQuiesceLeaderWhileFollowersCatchingUp (16401 ms)
[ RUN ] TServerQuiescingITest.TestQuiescingToolBasics
I20250904 13:57:54.433135 10634 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.98.190:35691
I20250904 13:57:54.434501 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:54.440603 12939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:54.440728 12940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:54.441365 12942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:54.443535 10634 server_base.cc:1047] running on GCE node
I20250904 13:57:54.444406 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:54.444602 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:54.444726 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994274444715 us; error 0 us; skew 500 ppm
I20250904 13:57:54.445171 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:54.447806 10634 webserver.cc:480] Webserver started at http://127.10.98.190:43093/ using document root <none> and password file <none>
I20250904 13:57:54.448289 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:54.448450 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:54.448678 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:54.449852 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingToolBasics.1756994241738102-10634-0/minicluster-data/master-0-root/instance:
uuid: "2fca1522d30747529781a81fe0d9eec2"
format_stamp: "Formatted at 2025-09-04 13:57:54 on dist-test-slave-jx76"
I20250904 13:57:54.454811 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20250904 13:57:54.458441 12947 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:54.459244 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250904 13:57:54.459535 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingToolBasics.1756994241738102-10634-0/minicluster-data/master-0-root
uuid: "2fca1522d30747529781a81fe0d9eec2"
format_stamp: "Formatted at 2025-09-04 13:57:54 on dist-test-slave-jx76"
I20250904 13:57:54.459853 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingToolBasics.1756994241738102-10634-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingToolBasics.1756994241738102-10634-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingToolBasics.1756994241738102-10634-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:54.490808 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:54.492125 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:54.532744 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.190:35691
I20250904 13:57:54.532887 12998 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.190:35691 every 8 connection(s)
I20250904 13:57:54.537343 12999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:54.549103 12999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2: Bootstrap starting.
I20250904 13:57:54.554004 12999 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:54.558614 12999 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2: No bootstrap required, opened a new log
I20250904 13:57:54.560724 12999 raft_consensus.cc:357] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fca1522d30747529781a81fe0d9eec2" member_type: VOTER }
I20250904 13:57:54.561162 12999 raft_consensus.cc:383] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:54.561416 12999 raft_consensus.cc:738] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fca1522d30747529781a81fe0d9eec2, State: Initialized, Role: FOLLOWER
I20250904 13:57:54.562007 12999 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fca1522d30747529781a81fe0d9eec2" member_type: VOTER }
I20250904 13:57:54.562546 12999 raft_consensus.cc:397] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:57:54.562776 12999 raft_consensus.cc:491] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250904 13:57:54.563059 12999 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:54.567880 12999 raft_consensus.cc:513] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fca1522d30747529781a81fe0d9eec2" member_type: VOTER }
I20250904 13:57:54.568435 12999 leader_election.cc:304] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2fca1522d30747529781a81fe0d9eec2; no voters:
I20250904 13:57:54.569671 12999 leader_election.cc:290] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:57:54.570048 13002 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:54.571448 13002 raft_consensus.cc:695] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2 [term 1 LEADER]: Becoming Leader. State: Replica: 2fca1522d30747529781a81fe0d9eec2, State: Running, Role: LEADER
I20250904 13:57:54.572172 13002 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fca1522d30747529781a81fe0d9eec2" member_type: VOTER }
I20250904 13:57:54.572718 12999 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 13:57:54.577502 13004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2fca1522d30747529781a81fe0d9eec2. Latest consensus state: current_term: 1 leader_uuid: "2fca1522d30747529781a81fe0d9eec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fca1522d30747529781a81fe0d9eec2" member_type: VOTER } }
I20250904 13:57:54.577723 13003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2fca1522d30747529781a81fe0d9eec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2fca1522d30747529781a81fe0d9eec2" member_type: VOTER } }
I20250904 13:57:54.578631 13004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2 [sys.catalog]: This master's current role is: LEADER
I20250904 13:57:54.578681 13003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2 [sys.catalog]: This master's current role is: LEADER
I20250904 13:57:54.582293 13010 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 13:57:54.587357 13010 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 13:57:54.590727 10634 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 13:57:54.596684 13010 catalog_manager.cc:1349] Generated new cluster ID: 70c0caad2af54625861f50b828886bc9
I20250904 13:57:54.597047 13010 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 13:57:54.613317 13010 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 13:57:54.614755 13010 catalog_manager.cc:1506] Loading token signing keys...
I20250904 13:57:54.627315 13010 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2: Generated new TSK 0
I20250904 13:57:54.627995 13010 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 13:57:54.658526 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:57:54.665050 13020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:54.666333 13021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:57:54.667636 13023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:57:54.668278 10634 server_base.cc:1047] running on GCE node
I20250904 13:57:54.669088 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:57:54.669303 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:57:54.669454 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994274669435 us; error 0 us; skew 500 ppm
I20250904 13:57:54.669957 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:57:54.672346 10634 webserver.cc:480] Webserver started at http://127.10.98.129:40331/ using document root <none> and password file <none>
I20250904 13:57:54.672878 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:57:54.673072 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:57:54.673336 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:57:54.674685 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingToolBasics.1756994241738102-10634-0/minicluster-data/ts-0-root/instance:
uuid: "a29667e794954beb9acabde8b1362203"
format_stamp: "Formatted at 2025-09-04 13:57:54 on dist-test-slave-jx76"
I20250904 13:57:54.679636 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250904 13:57:54.683204 13028 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:54.683981 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250904 13:57:54.684299 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingToolBasics.1756994241738102-10634-0/minicluster-data/ts-0-root
uuid: "a29667e794954beb9acabde8b1362203"
format_stamp: "Formatted at 2025-09-04 13:57:54 on dist-test-slave-jx76"
I20250904 13:57:54.684597 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingToolBasics.1756994241738102-10634-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingToolBasics.1756994241738102-10634-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiescingToolBasics.1756994241738102-10634-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:57:54.709517 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:57:54.710824 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:57:54.712384 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:57:54.714840 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:57:54.715054 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:54.715298 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:57:54.715461 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:57:54.758368 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.129:39697
I20250904 13:57:54.758467 13090 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.129:39697 every 8 connection(s)
I20250904 13:57:54.779955 13091 heartbeater.cc:344] Connected to a master server at 127.10.98.190:35691
I20250904 13:57:54.780377 13091 heartbeater.cc:461] Registering TS with master...
I20250904 13:57:54.781199 13091 heartbeater.cc:507] Master 127.10.98.190:35691 requested a full tablet report, sending...
I20250904 13:57:54.783279 12964 ts_manager.cc:194] Registered new tserver with Master: a29667e794954beb9acabde8b1362203 (127.10.98.129:39697)
I20250904 13:57:54.783917 10634 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.022039961s
I20250904 13:57:54.785329 12964 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55172
I20250904 13:57:54.818101 10634 test_util.cc:276] Using random seed: 463435802
I20250904 13:57:54.841557 12964 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55208:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250904 13:57:54.878665 13056 tablet_service.cc:1468] Processing CreateTablet for tablet 4d6054ee29d242bfb48451e08f899109 (DEFAULT_TABLE table=test-workload [id=58c7ac42664242e092e917944da77344]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:57:54.879791 13056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d6054ee29d242bfb48451e08f899109. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:57:54.895131 13109 tablet_bootstrap.cc:492] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203: Bootstrap starting.
I20250904 13:57:54.900058 13109 tablet_bootstrap.cc:654] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203: Neither blocks nor log segments found. Creating new log.
I20250904 13:57:54.904623 13109 tablet_bootstrap.cc:492] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203: No bootstrap required, opened a new log
I20250904 13:57:54.905040 13109 ts_tablet_manager.cc:1397] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20250904 13:57:54.907199 13109 raft_consensus.cc:357] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29667e794954beb9acabde8b1362203" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 39697 } }
I20250904 13:57:54.907616 13109 raft_consensus.cc:383] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:57:54.907863 13109 raft_consensus.cc:738] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a29667e794954beb9acabde8b1362203, State: Initialized, Role: FOLLOWER
I20250904 13:57:54.908398 13109 consensus_queue.cc:260] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29667e794954beb9acabde8b1362203" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 39697 } }
I20250904 13:57:54.908856 13109 raft_consensus.cc:397] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:57:54.909083 13109 raft_consensus.cc:491] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250904 13:57:54.909341 13109 raft_consensus.cc:3058] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:57:54.914774 13109 raft_consensus.cc:513] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29667e794954beb9acabde8b1362203" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 39697 } }
I20250904 13:57:54.915380 13109 leader_election.cc:304] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a29667e794954beb9acabde8b1362203; no voters:
I20250904 13:57:54.916769 13109 leader_election.cc:290] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:57:54.917050 13111 raft_consensus.cc:2802] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:57:54.918561 13111 raft_consensus.cc:695] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203 [term 1 LEADER]: Becoming Leader. State: Replica: a29667e794954beb9acabde8b1362203, State: Running, Role: LEADER
I20250904 13:57:54.918720 13091 heartbeater.cc:499] Master 127.10.98.190:35691 was elected leader, sending a full tablet report...
I20250904 13:57:54.919510 13109 ts_tablet_manager.cc:1428] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203: Time spent starting tablet: real 0.014s user 0.014s sys 0.002s
I20250904 13:57:54.919420 13111 consensus_queue.cc:237] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29667e794954beb9acabde8b1362203" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 39697 } }
I20250904 13:57:54.927834 12964 catalog_manager.cc:5582] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203 reported cstate change: term changed from 0 to 1, leader changed from <none> to a29667e794954beb9acabde8b1362203 (127.10.98.129). New cstate: current_term: 1 leader_uuid: "a29667e794954beb9acabde8b1362203" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29667e794954beb9acabde8b1362203" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 39697 } health_report { overall_health: HEALTHY } } }
I20250904 13:57:56.617594 13056 tablet_service.cc:1423] Tablet server a29667e794954beb9acabde8b1362203 set to quiescing
W20250904 13:57:58.015043 13129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250904 13:57:58.015780 13129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250904 13:57:58.048969 13129 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
W20250904 13:57:59.263425 13129 thread.cc:641] rpc reactor (reactor) Time spent creating pthread: real 1.170s user 0.401s sys 0.765s
W20250904 13:57:59.263753 13129 thread.cc:608] rpc reactor (reactor) Time spent starting thread: real 1.170s user 0.401s sys 0.765s
I20250904 13:57:59.293473 13056 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
W20250904 13:58:00.700531 13142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250904 13:58:00.701253 13142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250904 13:58:00.734638 13142 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
W20250904 13:58:01.960244 13142 thread.cc:641] rpc reactor (reactor) Time spent creating pthread: real 1.182s user 0.494s sys 0.687s
W20250904 13:58:01.960705 13142 thread.cc:608] rpc reactor (reactor) Time spent starting thread: real 1.183s user 0.494s sys 0.687s
I20250904 13:58:02.243003 13056 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250904 13:58:05.074807 13056 tablet_service.cc:1423] Tablet server a29667e794954beb9acabde8b1362203 set to quiescing
W20250904 13:58:06.468937 13184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250904 13:58:06.469638 13184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250904 13:58:06.503206 13184 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
W20250904 13:58:07.743633 13184 thread.cc:641] rpc reactor (reactor) Time spent creating pthread: real 1.198s user 0.506s sys 0.690s
W20250904 13:58:07.744104 13184 thread.cc:608] rpc reactor (reactor) Time spent starting thread: real 1.198s user 0.506s sys 0.690s
I20250904 13:58:07.774638 13056 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
W20250904 13:58:09.167842 13197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250904 13:58:09.168561 13197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250904 13:58:09.202029 13197 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
W20250904 13:58:10.455753 13197 thread.cc:641] rpc reactor (reactor) Time spent creating pthread: real 1.210s user 0.477s sys 0.731s
W20250904 13:58:10.456254 13197 thread.cc:608] rpc reactor (reactor) Time spent starting thread: real 1.211s user 0.477s sys 0.731s
I20250904 13:58:10.795603 13056 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
W20250904 13:58:12.244879 13087 debug-util.cc:398] Leaking SignalData structure 0x7b08000c1460 after lost signal to thread 10637
W20250904 13:58:12.247534 13087 debug-util.cc:398] Leaking SignalData structure 0x7b08000b7a00 after lost signal to thread 12998
W20250904 13:58:12.250540 13087 debug-util.cc:398] Leaking SignalData structure 0x7b0800179e00 after lost signal to thread 13090
I20250904 13:58:14.154978 13056 tablet_service.cc:1423] Tablet server a29667e794954beb9acabde8b1362203 set to not quiescing
W20250904 13:58:15.563473 13239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250904 13:58:15.564165 13239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250904 13:58:15.597556 13239 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
W20250904 13:58:16.808342 13239 thread.cc:641] rpc reactor (reactor) Time spent creating pthread: real 1.167s user 0.486s sys 0.678s
W20250904 13:58:16.808689 13239 thread.cc:608] rpc reactor (reactor) Time spent starting thread: real 1.167s user 0.486s sys 0.678s
I20250904 13:58:16.848919 13056 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
W20250904 13:58:18.245774 13252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250904 13:58:18.246500 13252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250904 13:58:18.279799 13252 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
W20250904 13:58:19.505645 13252 thread.cc:641] rpc reactor (reactor) Time spent creating pthread: real 1.182s user 0.560s sys 0.618s
W20250904 13:58:19.506145 13252 thread.cc:608] rpc reactor (reactor) Time spent starting thread: real 1.182s user 0.560s sys 0.618s
I20250904 13:58:19.796278 13056 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
W20250904 13:58:21.339385 13280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250904 13:58:21.340143 13280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250904 13:58:21.373553 13280 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
W20250904 13:58:22.593706 13280 thread.cc:641] rpc reactor (reactor) Time spent creating pthread: real 1.177s user 0.502s sys 0.672s
W20250904 13:58:22.594095 13280 thread.cc:608] rpc reactor (reactor) Time spent starting thread: real 1.178s user 0.502s sys 0.672s
W20250904 13:58:24.656437 13313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250904 13:58:24.657418 13313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250904 13:58:24.718694 13313 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
W20250904 13:58:26.189922 13327 debug-util.cc:398] Leaking SignalData structure 0x7b08000373e0 after lost signal to thread 13313
W20250904 13:58:26.190739 13327 kernel_stack_watchdog.cc:198] Thread 13313 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 387ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20250904 13:58:26.527666 13313 thread.cc:641] rpc reactor (reactor) Time spent creating pthread: real 1.727s user 0.003s sys 0.000s
W20250904 13:58:26.725225 13313 thread.cc:608] rpc reactor (reactor) Time spent starting thread: real 1.925s user 0.004s sys 0.016s
I20250904 13:58:26.793569 13056 tablet_service.cc:1430] Tablet server has 1 leaders and 1 scanners
W20250904 13:58:27.392907 13327 debug-util.cc:398] Leaking SignalData structure 0x7b08000347c0 after lost signal to thread 13313
W20250904 13:58:28.533077 13336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250904 13:58:28.534068 13336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250904 13:58:28.603791 13336 flags.cc:425] Enabled experimental flag: --enable_multi_tenancy=false
W20250904 13:58:30.101356 13342 debug-util.cc:398] Leaking SignalData structure 0x7b08000347a0 after lost signal to thread 13336
W20250904 13:58:30.101955 13342 kernel_stack_watchdog.cc:198] Thread 13336 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:642 for 401ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
W20250904 13:58:30.543752 13336 thread.cc:641] rpc reactor (reactor) Time spent creating pthread: real 1.845s user 0.519s sys 0.877s
W20250904 13:58:30.726048 13336 thread.cc:608] rpc reactor (reactor) Time spent starting thread: real 2.028s user 0.521s sys 0.892s
I20250904 13:58:31.281097 13056 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
W20250904 13:58:31.303826 13342 debug-util.cc:398] Leaking SignalData structure 0x7b08000379e0 after lost signal to thread 13336
I20250904 13:58:34.882521 13056 tablet_service.cc:1423] Tablet server a29667e794954beb9acabde8b1362203 set to quiescing
I20250904 13:58:34.885290 13056 tablet_service.cc:1430] Tablet server has 1 leaders and 0 scanners
I20250904 13:58:35.923053 10634 tablet_server.cc:178] TabletServer@127.10.98.129:0 shutting down...
I20250904 13:58:35.979120 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:58:35.980041 10634 tablet_replica.cc:331] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203: stopping tablet replica
I20250904 13:58:35.981004 10634 raft_consensus.cc:2241] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:58:35.982285 10634 raft_consensus.cc:2270] T 4d6054ee29d242bfb48451e08f899109 P a29667e794954beb9acabde8b1362203 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:58:36.057045 10634 tablet_server.cc:195] TabletServer@127.10.98.129:0 shutdown complete.
I20250904 13:58:36.077070 10634 master.cc:561] Master@127.10.98.190:35691 shutting down...
I20250904 13:58:36.107964 10634 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:58:36.108780 10634 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:58:36.109197 10634 tablet_replica.cc:331] T 00000000000000000000000000000000 P 2fca1522d30747529781a81fe0d9eec2: stopping tablet replica
I20250904 13:58:36.130309 10634 master.cc:583] Master@127.10.98.190:35691 shutdown complete.
[ OK ] TServerQuiescingITest.TestQuiescingToolBasics (41736 ms)
[ RUN ] TServerQuiescingITest.TestQuiesceAndStopTool
I20250904 13:58:36.171007 10634 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.98.190:44159
I20250904 13:58:36.172567 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:58:36.179574 13380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:58:36.179764 13381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:58:36.188460 13383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:58:36.188683 10634 server_base.cc:1047] running on GCE node
I20250904 13:58:36.190510 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:58:36.190822 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:58:36.191002 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994316190981 us; error 0 us; skew 500 ppm
I20250904 13:58:36.191706 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:58:36.194764 10634 webserver.cc:480] Webserver started at http://127.10.98.190:45583/ using document root <none> and password file <none>
I20250904 13:58:36.195497 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:58:36.195739 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:58:36.196060 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:58:36.197540 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceAndStopTool.1756994241738102-10634-0/minicluster-data/master-0-root/instance:
uuid: "f0a09fc7740b4822bd5175fe9d4f6748"
format_stamp: "Formatted at 2025-09-04 13:58:36 on dist-test-slave-jx76"
I20250904 13:58:36.203464 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20250904 13:58:36.207823 13388 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:58:36.208801 10634 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20250904 13:58:36.209103 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceAndStopTool.1756994241738102-10634-0/minicluster-data/master-0-root
uuid: "f0a09fc7740b4822bd5175fe9d4f6748"
format_stamp: "Formatted at 2025-09-04 13:58:36 on dist-test-slave-jx76"
I20250904 13:58:36.209405 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceAndStopTool.1756994241738102-10634-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceAndStopTool.1756994241738102-10634-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceAndStopTool.1756994241738102-10634-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:58:36.242519 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:58:36.244035 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:58:36.292084 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.190:44159
I20250904 13:58:36.292172 13439 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.190:44159 every 8 connection(s)
I20250904 13:58:36.297107 13440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:58:36.310937 13440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748: Bootstrap starting.
I20250904 13:58:36.316691 13440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748: Neither blocks nor log segments found. Creating new log.
I20250904 13:58:36.321640 13440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748: No bootstrap required, opened a new log
I20250904 13:58:36.324090 13440 raft_consensus.cc:357] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a09fc7740b4822bd5175fe9d4f6748" member_type: VOTER }
I20250904 13:58:36.324551 13440 raft_consensus.cc:383] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:58:36.324820 13440 raft_consensus.cc:738] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0a09fc7740b4822bd5175fe9d4f6748, State: Initialized, Role: FOLLOWER
I20250904 13:58:36.325390 13440 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a09fc7740b4822bd5175fe9d4f6748" member_type: VOTER }
I20250904 13:58:36.325877 13440 raft_consensus.cc:397] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:58:36.326092 13440 raft_consensus.cc:491] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250904 13:58:36.326381 13440 raft_consensus.cc:3058] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:58:36.331728 13440 raft_consensus.cc:513] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a09fc7740b4822bd5175fe9d4f6748" member_type: VOTER }
I20250904 13:58:36.332306 13440 leader_election.cc:304] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0a09fc7740b4822bd5175fe9d4f6748; no voters:
I20250904 13:58:36.333658 13440 leader_election.cc:290] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:58:36.334172 13443 raft_consensus.cc:2802] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:58:36.335621 13443 raft_consensus.cc:695] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748 [term 1 LEADER]: Becoming Leader. State: Replica: f0a09fc7740b4822bd5175fe9d4f6748, State: Running, Role: LEADER
I20250904 13:58:36.336396 13443 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a09fc7740b4822bd5175fe9d4f6748" member_type: VOTER }
I20250904 13:58:36.336963 13440 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 13:58:36.342792 13444 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0a09fc7740b4822bd5175fe9d4f6748" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a09fc7740b4822bd5175fe9d4f6748" member_type: VOTER } }
I20250904 13:58:36.343755 13444 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748 [sys.catalog]: This master's current role is: LEADER
I20250904 13:58:36.343889 13445 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0a09fc7740b4822bd5175fe9d4f6748. Latest consensus state: current_term: 1 leader_uuid: "f0a09fc7740b4822bd5175fe9d4f6748" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a09fc7740b4822bd5175fe9d4f6748" member_type: VOTER } }
I20250904 13:58:36.344468 13445 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748 [sys.catalog]: This master's current role is: LEADER
I20250904 13:58:36.349223 13451 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 13:58:36.354312 13451 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 13:58:36.357227 10634 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 13:58:36.363361 13451 catalog_manager.cc:1349] Generated new cluster ID: a2258daaf8b5450690b36dd6f846cffa
I20250904 13:58:36.363638 13451 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 13:58:36.377411 13451 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 13:58:36.378917 13451 catalog_manager.cc:1506] Loading token signing keys...
I20250904 13:58:36.395035 13451 catalog_manager.cc:5955] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748: Generated new TSK 0
I20250904 13:58:36.395763 13451 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 13:58:36.424886 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:58:36.432040 13461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:58:36.433733 13462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:58:36.435587 13464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:58:36.436080 10634 server_base.cc:1047] running on GCE node
I20250904 13:58:36.437158 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:58:36.437372 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:58:36.437541 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994316437525 us; error 0 us; skew 500 ppm
I20250904 13:58:36.438263 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:58:36.440915 10634 webserver.cc:480] Webserver started at http://127.10.98.129:34681/ using document root <none> and password file <none>
I20250904 13:58:36.441505 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:58:36.441720 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:58:36.442003 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:58:36.443301 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceAndStopTool.1756994241738102-10634-0/minicluster-data/ts-0-root/instance:
uuid: "69dc5e3b237246b6bfaacc3b5384e10b"
format_stamp: "Formatted at 2025-09-04 13:58:36 on dist-test-slave-jx76"
I20250904 13:58:36.448490 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20250904 13:58:36.452374 13469 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:58:36.453207 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250904 13:58:36.453503 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceAndStopTool.1756994241738102-10634-0/minicluster-data/ts-0-root
uuid: "69dc5e3b237246b6bfaacc3b5384e10b"
format_stamp: "Formatted at 2025-09-04 13:58:36 on dist-test-slave-jx76"
I20250904 13:58:36.453804 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceAndStopTool.1756994241738102-10634-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceAndStopTool.1756994241738102-10634-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceAndStopTool.1756994241738102-10634-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:58:36.474347 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:58:36.475692 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:58:36.477442 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:58:36.480123 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:58:36.480345 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:58:36.480645 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:58:36.480808 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:58:36.529186 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.129:35645
I20250904 13:58:36.529302 13531 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.129:35645 every 8 connection(s)
I20250904 13:58:36.534492 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:58:36.543268 13536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:58:36.546512 13537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:58:36.547026 13539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:58:36.547920 10634 server_base.cc:1047] running on GCE node
I20250904 13:58:36.548952 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:58:36.549150 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:58:36.549335 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994316549319 us; error 0 us; skew 500 ppm
I20250904 13:58:36.549853 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:58:36.552974 10634 webserver.cc:480] Webserver started at http://127.10.98.130:33823/ using document root <none> and password file <none>
I20250904 13:58:36.553579 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:58:36.553623 13532 heartbeater.cc:344] Connected to a master server at 127.10.98.190:44159
I20250904 13:58:36.553815 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:58:36.554073 13532 heartbeater.cc:461] Registering TS with master...
I20250904 13:58:36.554204 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:58:36.555001 13532 heartbeater.cc:507] Master 127.10.98.190:44159 requested a full tablet report, sending...
I20250904 13:58:36.555641 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceAndStopTool.1756994241738102-10634-0/minicluster-data/ts-1-root/instance:
uuid: "9ab118fc96be4284a051212d6c9e58b2"
format_stamp: "Formatted at 2025-09-04 13:58:36 on dist-test-slave-jx76"
I20250904 13:58:36.557152 13405 ts_manager.cc:194] Registered new tserver with Master: 69dc5e3b237246b6bfaacc3b5384e10b (127.10.98.129:35645)
I20250904 13:58:36.559231 13405 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41530
I20250904 13:58:36.561187 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20250904 13:58:36.565395 13544 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:58:36.566342 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250904 13:58:36.566635 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceAndStopTool.1756994241738102-10634-0/minicluster-data/ts-1-root
uuid: "9ab118fc96be4284a051212d6c9e58b2"
format_stamp: "Formatted at 2025-09-04 13:58:36 on dist-test-slave-jx76"
I20250904 13:58:36.566902 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceAndStopTool.1756994241738102-10634-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceAndStopTool.1756994241738102-10634-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceAndStopTool.1756994241738102-10634-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:58:36.587503 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:58:36.588831 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:58:36.590502 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:58:36.593058 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:58:36.593248 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:58:36.593509 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:58:36.593659 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:58:36.645139 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.130:41157
I20250904 13:58:36.645249 13606 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.130:41157 every 8 connection(s)
I20250904 13:58:36.650702 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:58:36.657896 13610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:58:36.660681 13611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:58:36.666651 13607 heartbeater.cc:344] Connected to a master server at 127.10.98.190:44159
I20250904 13:58:36.667168 13607 heartbeater.cc:461] Registering TS with master...
I20250904 13:58:36.667268 10634 server_base.cc:1047] running on GCE node
W20250904 13:58:36.667372 13613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:58:36.668444 13607 heartbeater.cc:507] Master 127.10.98.190:44159 requested a full tablet report, sending...
I20250904 13:58:36.668569 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:58:36.668902 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:58:36.669116 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994316669098 us; error 0 us; skew 500 ppm
I20250904 13:58:36.669780 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:58:36.671149 13405 ts_manager.cc:194] Registered new tserver with Master: 9ab118fc96be4284a051212d6c9e58b2 (127.10.98.130:41157)
I20250904 13:58:36.672972 10634 webserver.cc:480] Webserver started at http://127.10.98.131:45903/ using document root <none> and password file <none>
I20250904 13:58:36.673343 13405 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41534
I20250904 13:58:36.673713 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:58:36.674000 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:58:36.674374 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:58:36.675761 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceAndStopTool.1756994241738102-10634-0/minicluster-data/ts-2-root/instance:
uuid: "ccb04f45a1ee4e0e8942995609d0b792"
format_stamp: "Formatted at 2025-09-04 13:58:36 on dist-test-slave-jx76"
I20250904 13:58:36.680729 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20250904 13:58:36.686017 13618 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:58:36.687086 10634 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20250904 13:58:36.687541 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceAndStopTool.1756994241738102-10634-0/minicluster-data/ts-2-root
uuid: "ccb04f45a1ee4e0e8942995609d0b792"
format_stamp: "Formatted at 2025-09-04 13:58:36 on dist-test-slave-jx76"
I20250904 13:58:36.688004 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceAndStopTool.1756994241738102-10634-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceAndStopTool.1756994241738102-10634-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.TServerQuiescingITest.TestQuiesceAndStopTool.1756994241738102-10634-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:58:36.706653 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:58:36.708119 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:58:36.709590 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:58:36.712281 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:58:36.712517 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:58:36.712774 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:58:36.712925 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:58:36.761711 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.131:39811
I20250904 13:58:36.761858 13680 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.131:39811 every 8 connection(s)
I20250904 13:58:36.776908 13681 heartbeater.cc:344] Connected to a master server at 127.10.98.190:44159
I20250904 13:58:36.777360 13681 heartbeater.cc:461] Registering TS with master...
I20250904 13:58:36.778223 13681 heartbeater.cc:507] Master 127.10.98.190:44159 requested a full tablet report, sending...
I20250904 13:58:36.780510 13405 ts_manager.cc:194] Registered new tserver with Master: ccb04f45a1ee4e0e8942995609d0b792 (127.10.98.131:39811)
I20250904 13:58:36.780618 10634 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015312464s
I20250904 13:58:36.782068 13405 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41538
I20250904 13:58:36.818202 10634 test_util.cc:276] Using random seed: 505435897
I20250904 13:58:36.844198 13405 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41564:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 13:58:36.846562 13405 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 13:58:36.889935 13497 tablet_service.cc:1468] Processing CreateTablet for tablet 1bf8c99aa65744d79f0e7b0524e57e76 (DEFAULT_TABLE table=test-workload [id=c6ede9e018454154bc0bd73ba168c23a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:58:36.891472 13497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bf8c99aa65744d79f0e7b0524e57e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:58:36.901100 13572 tablet_service.cc:1468] Processing CreateTablet for tablet 1bf8c99aa65744d79f0e7b0524e57e76 (DEFAULT_TABLE table=test-workload [id=c6ede9e018454154bc0bd73ba168c23a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:58:36.901394 13646 tablet_service.cc:1468] Processing CreateTablet for tablet 1bf8c99aa65744d79f0e7b0524e57e76 (DEFAULT_TABLE table=test-workload [id=c6ede9e018454154bc0bd73ba168c23a]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:58:36.902360 13572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bf8c99aa65744d79f0e7b0524e57e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:58:36.902757 13646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bf8c99aa65744d79f0e7b0524e57e76. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:58:36.913260 13702 tablet_bootstrap.cc:492] T 1bf8c99aa65744d79f0e7b0524e57e76 P 69dc5e3b237246b6bfaacc3b5384e10b: Bootstrap starting.
I20250904 13:58:36.920992 13702 tablet_bootstrap.cc:654] T 1bf8c99aa65744d79f0e7b0524e57e76 P 69dc5e3b237246b6bfaacc3b5384e10b: Neither blocks nor log segments found. Creating new log.
I20250904 13:58:36.925302 13703 tablet_bootstrap.cc:492] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792: Bootstrap starting.
I20250904 13:58:36.928318 13704 tablet_bootstrap.cc:492] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2: Bootstrap starting.
I20250904 13:58:36.929072 13702 tablet_bootstrap.cc:492] T 1bf8c99aa65744d79f0e7b0524e57e76 P 69dc5e3b237246b6bfaacc3b5384e10b: No bootstrap required, opened a new log
I20250904 13:58:36.929757 13702 ts_tablet_manager.cc:1397] T 1bf8c99aa65744d79f0e7b0524e57e76 P 69dc5e3b237246b6bfaacc3b5384e10b: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.004s
I20250904 13:58:36.933318 13703 tablet_bootstrap.cc:654] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792: Neither blocks nor log segments found. Creating new log.
I20250904 13:58:36.932968 13702 raft_consensus.cc:357] T 1bf8c99aa65744d79f0e7b0524e57e76 P 69dc5e3b237246b6bfaacc3b5384e10b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dc5e3b237246b6bfaacc3b5384e10b" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35645 } } peers { permanent_uuid: "ccb04f45a1ee4e0e8942995609d0b792" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39811 } } peers { permanent_uuid: "9ab118fc96be4284a051212d6c9e58b2" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41157 } }
I20250904 13:58:36.933825 13702 raft_consensus.cc:383] T 1bf8c99aa65744d79f0e7b0524e57e76 P 69dc5e3b237246b6bfaacc3b5384e10b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:58:36.934219 13702 raft_consensus.cc:738] T 1bf8c99aa65744d79f0e7b0524e57e76 P 69dc5e3b237246b6bfaacc3b5384e10b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69dc5e3b237246b6bfaacc3b5384e10b, State: Initialized, Role: FOLLOWER
I20250904 13:58:36.934950 13704 tablet_bootstrap.cc:654] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2: Neither blocks nor log segments found. Creating new log.
I20250904 13:58:36.935094 13702 consensus_queue.cc:260] T 1bf8c99aa65744d79f0e7b0524e57e76 P 69dc5e3b237246b6bfaacc3b5384e10b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dc5e3b237246b6bfaacc3b5384e10b" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35645 } } peers { permanent_uuid: "ccb04f45a1ee4e0e8942995609d0b792" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39811 } } peers { permanent_uuid: "9ab118fc96be4284a051212d6c9e58b2" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41157 } }
I20250904 13:58:36.938030 13532 heartbeater.cc:499] Master 127.10.98.190:44159 was elected leader, sending a full tablet report...
I20250904 13:58:36.938769 13702 ts_tablet_manager.cc:1428] T 1bf8c99aa65744d79f0e7b0524e57e76 P 69dc5e3b237246b6bfaacc3b5384e10b: Time spent starting tablet: real 0.009s user 0.006s sys 0.004s
I20250904 13:58:36.941015 13704 tablet_bootstrap.cc:492] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2: No bootstrap required, opened a new log
I20250904 13:58:36.941471 13704 ts_tablet_manager.cc:1397] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20250904 13:58:36.942230 13703 tablet_bootstrap.cc:492] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792: No bootstrap required, opened a new log
I20250904 13:58:36.942739 13703 ts_tablet_manager.cc:1397] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.000s
I20250904 13:58:36.943929 13704 raft_consensus.cc:357] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dc5e3b237246b6bfaacc3b5384e10b" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35645 } } peers { permanent_uuid: "ccb04f45a1ee4e0e8942995609d0b792" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39811 } } peers { permanent_uuid: "9ab118fc96be4284a051212d6c9e58b2" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41157 } }
I20250904 13:58:36.944814 13704 raft_consensus.cc:383] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:58:36.945186 13704 raft_consensus.cc:738] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ab118fc96be4284a051212d6c9e58b2, State: Initialized, Role: FOLLOWER
I20250904 13:58:36.946630 13703 raft_consensus.cc:357] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dc5e3b237246b6bfaacc3b5384e10b" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35645 } } peers { permanent_uuid: "ccb04f45a1ee4e0e8942995609d0b792" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39811 } } peers { permanent_uuid: "9ab118fc96be4284a051212d6c9e58b2" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41157 } }
I20250904 13:58:36.947513 13703 raft_consensus.cc:383] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:58:36.946095 13704 consensus_queue.cc:260] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dc5e3b237246b6bfaacc3b5384e10b" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35645 } } peers { permanent_uuid: "ccb04f45a1ee4e0e8942995609d0b792" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39811 } } peers { permanent_uuid: "9ab118fc96be4284a051212d6c9e58b2" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41157 } }
I20250904 13:58:36.947839 13703 raft_consensus.cc:738] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccb04f45a1ee4e0e8942995609d0b792, State: Initialized, Role: FOLLOWER
I20250904 13:58:36.948691 13703 consensus_queue.cc:260] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dc5e3b237246b6bfaacc3b5384e10b" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35645 } } peers { permanent_uuid: "ccb04f45a1ee4e0e8942995609d0b792" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39811 } } peers { permanent_uuid: "9ab118fc96be4284a051212d6c9e58b2" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41157 } }
I20250904 13:58:36.950080 13607 heartbeater.cc:499] Master 127.10.98.190:44159 was elected leader, sending a full tablet report...
I20250904 13:58:36.950699 13704 ts_tablet_manager.cc:1428] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2: Time spent starting tablet: real 0.009s user 0.004s sys 0.005s
I20250904 13:58:36.953209 13681 heartbeater.cc:499] Master 127.10.98.190:44159 was elected leader, sending a full tablet report...
I20250904 13:58:36.953994 13703 ts_tablet_manager.cc:1428] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
I20250904 13:58:36.972268 13710 raft_consensus.cc:491] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:58:36.972826 13710 raft_consensus.cc:513] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dc5e3b237246b6bfaacc3b5384e10b" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35645 } } peers { permanent_uuid: "ccb04f45a1ee4e0e8942995609d0b792" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39811 } } peers { permanent_uuid: "9ab118fc96be4284a051212d6c9e58b2" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41157 } }
I20250904 13:58:36.975085 13710 leader_election.cc:290] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69dc5e3b237246b6bfaacc3b5384e10b (127.10.98.129:35645), 9ab118fc96be4284a051212d6c9e58b2 (127.10.98.130:41157)
I20250904 13:58:36.986061 13507 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1bf8c99aa65744d79f0e7b0524e57e76" candidate_uuid: "ccb04f45a1ee4e0e8942995609d0b792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69dc5e3b237246b6bfaacc3b5384e10b" is_pre_election: true
I20250904 13:58:36.986814 13507 raft_consensus.cc:2466] T 1bf8c99aa65744d79f0e7b0524e57e76 P 69dc5e3b237246b6bfaacc3b5384e10b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccb04f45a1ee4e0e8942995609d0b792 in term 0.
I20250904 13:58:36.986866 13582 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1bf8c99aa65744d79f0e7b0524e57e76" candidate_uuid: "ccb04f45a1ee4e0e8942995609d0b792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ab118fc96be4284a051212d6c9e58b2" is_pre_election: true
I20250904 13:58:36.987507 13582 raft_consensus.cc:2466] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ccb04f45a1ee4e0e8942995609d0b792 in term 0.
I20250904 13:58:36.988108 13621 leader_election.cc:304] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69dc5e3b237246b6bfaacc3b5384e10b, ccb04f45a1ee4e0e8942995609d0b792; no voters:
I20250904 13:58:36.988991 13710 raft_consensus.cc:2802] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:58:36.989272 13710 raft_consensus.cc:491] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:58:36.989579 13710 raft_consensus.cc:3058] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:58:36.995685 13710 raft_consensus.cc:513] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dc5e3b237246b6bfaacc3b5384e10b" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35645 } } peers { permanent_uuid: "ccb04f45a1ee4e0e8942995609d0b792" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39811 } } peers { permanent_uuid: "9ab118fc96be4284a051212d6c9e58b2" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41157 } }
I20250904 13:58:36.997421 13710 leader_election.cc:290] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [CANDIDATE]: Term 1 election: Requested vote from peers 69dc5e3b237246b6bfaacc3b5384e10b (127.10.98.129:35645), 9ab118fc96be4284a051212d6c9e58b2 (127.10.98.130:41157)
I20250904 13:58:36.998653 13507 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1bf8c99aa65744d79f0e7b0524e57e76" candidate_uuid: "ccb04f45a1ee4e0e8942995609d0b792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69dc5e3b237246b6bfaacc3b5384e10b"
I20250904 13:58:36.998653 13582 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1bf8c99aa65744d79f0e7b0524e57e76" candidate_uuid: "ccb04f45a1ee4e0e8942995609d0b792" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ab118fc96be4284a051212d6c9e58b2"
I20250904 13:58:36.999439 13507 raft_consensus.cc:3058] T 1bf8c99aa65744d79f0e7b0524e57e76 P 69dc5e3b237246b6bfaacc3b5384e10b [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:58:36.999488 13582 raft_consensus.cc:3058] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:58:37.008054 13507 raft_consensus.cc:2466] T 1bf8c99aa65744d79f0e7b0524e57e76 P 69dc5e3b237246b6bfaacc3b5384e10b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccb04f45a1ee4e0e8942995609d0b792 in term 1.
I20250904 13:58:37.008054 13582 raft_consensus.cc:2466] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ccb04f45a1ee4e0e8942995609d0b792 in term 1.
I20250904 13:58:37.009524 13621 leader_election.cc:304] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69dc5e3b237246b6bfaacc3b5384e10b, ccb04f45a1ee4e0e8942995609d0b792; no voters:
I20250904 13:58:37.010360 13710 raft_consensus.cc:2802] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:58:37.011693 13710 raft_consensus.cc:695] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 1 LEADER]: Becoming Leader. State: Replica: ccb04f45a1ee4e0e8942995609d0b792, State: Running, Role: LEADER
I20250904 13:58:37.012604 13710 consensus_queue.cc:237] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dc5e3b237246b6bfaacc3b5384e10b" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35645 } } peers { permanent_uuid: "ccb04f45a1ee4e0e8942995609d0b792" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39811 } } peers { permanent_uuid: "9ab118fc96be4284a051212d6c9e58b2" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41157 } }
W20250904 13:58:37.016291 13682 tablet.cc:2378] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250904 13:58:37.021371 13403 catalog_manager.cc:5582] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 reported cstate change: term changed from 0 to 1, leader changed from <none> to ccb04f45a1ee4e0e8942995609d0b792 (127.10.98.131). New cstate: current_term: 1 leader_uuid: "ccb04f45a1ee4e0e8942995609d0b792" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dc5e3b237246b6bfaacc3b5384e10b" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccb04f45a1ee4e0e8942995609d0b792" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ab118fc96be4284a051212d6c9e58b2" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41157 } health_report { overall_health: UNKNOWN } } }
I20250904 13:58:37.206822 13507 raft_consensus.cc:1273] T 1bf8c99aa65744d79f0e7b0524e57e76 P 69dc5e3b237246b6bfaacc3b5384e10b [term 1 FOLLOWER]: Refusing update from remote peer ccb04f45a1ee4e0e8942995609d0b792: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 13:58:37.206840 13582 raft_consensus.cc:1273] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [term 1 FOLLOWER]: Refusing update from remote peer ccb04f45a1ee4e0e8942995609d0b792: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 13:58:37.208729 13712 consensus_queue.cc:1035] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69dc5e3b237246b6bfaacc3b5384e10b" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250904 13:58:37.209424 13710 consensus_queue.cc:1035] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ab118fc96be4284a051212d6c9e58b2" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250904 13:58:37.226967 13636 tablet_service.cc:2912] Rejecting scan request for tablet 1bf8c99aa65744d79f0e7b0524e57e76: Uninitialized: clean time has not yet been initialized
W20250904 13:58:37.227021 13634 tablet_service.cc:2912] Rejecting scan request for tablet 1bf8c99aa65744d79f0e7b0524e57e76: Uninitialized: clean time has not yet been initialized
W20250904 13:58:37.231323 13635 tablet_service.cc:2912] Rejecting scan request for tablet 1bf8c99aa65744d79f0e7b0524e57e76: Uninitialized: clean time has not yet been initialized
I20250904 13:58:49.157352 13646 tablet_service.cc:1423] Tablet server ccb04f45a1ee4e0e8942995609d0b792 set to quiescing
I20250904 13:58:49.160316 13646 tablet_service.cc:1430] Tablet server has 1 leaders and 3 scanners
I20250904 13:58:49.161296 13743 raft_consensus.cc:991] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792: : Instructing follower 9ab118fc96be4284a051212d6c9e58b2 to start an election
I20250904 13:58:49.166208 13743 raft_consensus.cc:1079] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 1 LEADER]: Signalling peer 9ab118fc96be4284a051212d6c9e58b2 to start an election
I20250904 13:58:49.177402 13729 raft_consensus.cc:991] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792: : Instructing follower 9ab118fc96be4284a051212d6c9e58b2 to start an election
I20250904 13:58:49.177906 13729 raft_consensus.cc:1079] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 1 LEADER]: Signalling peer 9ab118fc96be4284a051212d6c9e58b2 to start an election
I20250904 13:58:49.179792 13581 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "1bf8c99aa65744d79f0e7b0524e57e76"
dest_uuid: "9ab118fc96be4284a051212d6c9e58b2"
from {username='slave'} at 127.0.0.1:38782
I20250904 13:58:49.180370 13581 raft_consensus.cc:491] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20250904 13:58:49.180822 13581 raft_consensus.cc:3058] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [term 1 FOLLOWER]: Advancing to term 2
I20250904 13:58:49.178113 13582 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "1bf8c99aa65744d79f0e7b0524e57e76"
dest_uuid: "9ab118fc96be4284a051212d6c9e58b2"
from {username='slave'} at 127.0.0.1:38782
I20250904 13:58:49.193420 13581 raft_consensus.cc:513] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dc5e3b237246b6bfaacc3b5384e10b" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35645 } } peers { permanent_uuid: "ccb04f45a1ee4e0e8942995609d0b792" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39811 } } peers { permanent_uuid: "9ab118fc96be4284a051212d6c9e58b2" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41157 } }
I20250904 13:58:49.195053 13580 raft_consensus.cc:1238] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [term 2 FOLLOWER]: Rejecting Update request from peer ccb04f45a1ee4e0e8942995609d0b792 for earlier term 1. Current term is 2. Ops: []
I20250904 13:58:49.198429 13582 raft_consensus.cc:491] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20250904 13:58:49.198947 13582 raft_consensus.cc:3058] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [term 2 FOLLOWER]: Advancing to term 3
I20250904 13:58:49.199373 13743 consensus_queue.cc:1046] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9ab118fc96be4284a051212d6c9e58b2" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41157 }, Status: INVALID_TERM, Last received: 1.314, Next index: 315, Last known committed idx: 313, Time since last communication: 0.000s
I20250904 13:58:49.203361 13773 raft_consensus.cc:3053] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 1 LEADER]: Stepping down as leader of term 1
I20250904 13:58:49.203740 13773 raft_consensus.cc:738] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: ccb04f45a1ee4e0e8942995609d0b792, State: Running, Role: LEADER
I20250904 13:58:49.204589 13773 consensus_queue.cc:260] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 314, Committed index: 314, Last appended: 1.314, Last appended by leader: 314, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dc5e3b237246b6bfaacc3b5384e10b" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35645 } } peers { permanent_uuid: "ccb04f45a1ee4e0e8942995609d0b792" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39811 } } peers { permanent_uuid: "9ab118fc96be4284a051212d6c9e58b2" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41157 } }
I20250904 13:58:49.205555 13773 raft_consensus.cc:3058] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 1 FOLLOWER]: Advancing to term 2
I20250904 13:58:49.226300 13581 leader_election.cc:290] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [CANDIDATE]: Term 2 election: Requested vote from peers 69dc5e3b237246b6bfaacc3b5384e10b (127.10.98.129:35645), ccb04f45a1ee4e0e8942995609d0b792 (127.10.98.131:39811)
W20250904 13:58:49.230082 13632 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:45034: Illegal state: replica ccb04f45a1ee4e0e8942995609d0b792 is not leader of this config: current role FOLLOWER
I20250904 13:58:49.229019 13582 raft_consensus.cc:513] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dc5e3b237246b6bfaacc3b5384e10b" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35645 } } peers { permanent_uuid: "ccb04f45a1ee4e0e8942995609d0b792" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39811 } } peers { permanent_uuid: "9ab118fc96be4284a051212d6c9e58b2" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41157 } }
I20250904 13:58:49.238098 13582 leader_election.cc:290] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [CANDIDATE]: Term 3 election: Requested vote from peers 69dc5e3b237246b6bfaacc3b5384e10b (127.10.98.129:35645), ccb04f45a1ee4e0e8942995609d0b792 (127.10.98.131:39811)
W20250904 13:58:49.242547 13631 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:45034: Illegal state: replica ccb04f45a1ee4e0e8942995609d0b792 is not leader of this config: current role FOLLOWER
I20250904 13:58:49.254730 13655 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1bf8c99aa65744d79f0e7b0524e57e76" candidate_uuid: "9ab118fc96be4284a051212d6c9e58b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 314 } } ignore_live_leader: true dest_uuid: "ccb04f45a1ee4e0e8942995609d0b792"
I20250904 13:58:49.255594 13655 raft_consensus.cc:3058] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 2 FOLLOWER]: Advancing to term 3
I20250904 13:58:49.260051 13656 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1bf8c99aa65744d79f0e7b0524e57e76" candidate_uuid: "9ab118fc96be4284a051212d6c9e58b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 314 } } ignore_live_leader: true dest_uuid: "ccb04f45a1ee4e0e8942995609d0b792"
I20250904 13:58:49.260761 13506 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1bf8c99aa65744d79f0e7b0524e57e76" candidate_uuid: "9ab118fc96be4284a051212d6c9e58b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 314 } } ignore_live_leader: true dest_uuid: "69dc5e3b237246b6bfaacc3b5384e10b"
I20250904 13:58:49.261106 13507 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "1bf8c99aa65744d79f0e7b0524e57e76" candidate_uuid: "9ab118fc96be4284a051212d6c9e58b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 314 } } ignore_live_leader: true dest_uuid: "69dc5e3b237246b6bfaacc3b5384e10b"
I20250904 13:58:49.261409 13506 raft_consensus.cc:3058] T 1bf8c99aa65744d79f0e7b0524e57e76 P 69dc5e3b237246b6bfaacc3b5384e10b [term 1 FOLLOWER]: Advancing to term 2
I20250904 13:58:49.271173 13506 raft_consensus.cc:2466] T 1bf8c99aa65744d79f0e7b0524e57e76 P 69dc5e3b237246b6bfaacc3b5384e10b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ab118fc96be4284a051212d6c9e58b2 in term 2.
I20250904 13:58:49.273007 13547 leader_election.cc:304] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 69dc5e3b237246b6bfaacc3b5384e10b, 9ab118fc96be4284a051212d6c9e58b2; no voters: ccb04f45a1ee4e0e8942995609d0b792
I20250904 13:58:49.283756 13780 raft_consensus.cc:2762] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20250904 13:58:49.288398 13655 raft_consensus.cc:2466] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ab118fc96be4284a051212d6c9e58b2 in term 3.
I20250904 13:58:49.291214 13548 leader_election.cc:304] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9ab118fc96be4284a051212d6c9e58b2, ccb04f45a1ee4e0e8942995609d0b792; no voters: 69dc5e3b237246b6bfaacc3b5384e10b
I20250904 13:58:49.292796 13780 raft_consensus.cc:2802] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [term 3 FOLLOWER]: Leader election won for term 3
W20250904 13:58:49.299813 13487 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37636: Illegal state: replica 69dc5e3b237246b6bfaacc3b5384e10b is not leader of this config: current role FOLLOWER
W20250904 13:58:49.300379 13486 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37636: Illegal state: replica 69dc5e3b237246b6bfaacc3b5384e10b is not leader of this config: current role FOLLOWER
I20250904 13:58:49.301126 13780 raft_consensus.cc:695] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [term 3 LEADER]: Becoming Leader. State: Replica: 9ab118fc96be4284a051212d6c9e58b2, State: Running, Role: LEADER
W20250904 13:58:49.304093 13485 tablet_service.cc:696] failed op from {username='slave'} at 127.0.0.1:37636: Illegal state: replica 69dc5e3b237246b6bfaacc3b5384e10b is not leader of this config: current role FOLLOWER
I20250904 13:58:49.305743 13780 consensus_queue.cc:237] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 313, Committed index: 313, Last appended: 1.314, Last appended by leader: 314, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dc5e3b237246b6bfaacc3b5384e10b" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35645 } } peers { permanent_uuid: "ccb04f45a1ee4e0e8942995609d0b792" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39811 } } peers { permanent_uuid: "9ab118fc96be4284a051212d6c9e58b2" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41157 } }
I20250904 13:58:49.321722 13397 catalog_manager.cc:5582] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 reported cstate change: term changed from 1 to 3, leader changed from ccb04f45a1ee4e0e8942995609d0b792 (127.10.98.131) to 9ab118fc96be4284a051212d6c9e58b2 (127.10.98.130). New cstate: current_term: 3 leader_uuid: "9ab118fc96be4284a051212d6c9e58b2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69dc5e3b237246b6bfaacc3b5384e10b" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ccb04f45a1ee4e0e8942995609d0b792" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ab118fc96be4284a051212d6c9e58b2" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41157 } health_report { overall_health: HEALTHY } } }
I20250904 13:58:49.353314 13506 raft_consensus.cc:3058] T 1bf8c99aa65744d79f0e7b0524e57e76 P 69dc5e3b237246b6bfaacc3b5384e10b [term 2 FOLLOWER]: Advancing to term 3
I20250904 13:58:49.353399 13655 raft_consensus.cc:1273] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 3 FOLLOWER]: Refusing update from remote peer 9ab118fc96be4284a051212d6c9e58b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 314. Preceding OpId from leader: term: 3 index: 316. (index mismatch)
I20250904 13:58:49.356109 13781 consensus_queue.cc:1035] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb04f45a1ee4e0e8942995609d0b792" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 39811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 315, Last known committed idx: 314, Time since last communication: 0.001s
I20250904 13:58:49.365496 13506 raft_consensus.cc:1273] T 1bf8c99aa65744d79f0e7b0524e57e76 P 69dc5e3b237246b6bfaacc3b5384e10b [term 3 FOLLOWER]: Refusing update from remote peer 9ab118fc96be4284a051212d6c9e58b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 314. Preceding OpId from leader: term: 3 index: 316. (index mismatch)
I20250904 13:58:49.373071 13781 consensus_queue.cc:1035] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69dc5e3b237246b6bfaacc3b5384e10b" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 315, Last known committed idx: 313, Time since last communication: 0.001s
W20250904 13:58:52.411667 13716 meta_cache.cc:1261] Time spent looking up entry by key: real 0.081s user 0.001s sys 0.003s
I20250904 13:58:55.374363 13646 tablet_service.cc:1423] Tablet server ccb04f45a1ee4e0e8942995609d0b792 set to quiescing
I20250904 13:58:55.377933 13646 tablet_service.cc:1430] Tablet server has 0 leaders and 0 scanners
W20250904 13:58:56.580595 13714 meta_cache.cc:1261] Time spent looking up entry by key: real 0.054s user 0.000s sys 0.000s
I20250904 13:59:00.794489 13646 tablet_service.cc:1423] Tablet server ccb04f45a1ee4e0e8942995609d0b792 set to not quiescing
W20250904 13:59:01.305032 13677 debug-util.cc:398] Leaking SignalData structure 0x7b0800120040 after lost signal to thread 10637
W20250904 13:59:01.306404 13677 debug-util.cc:398] Leaking SignalData structure 0x7b0800113ce0 after lost signal to thread 13439
W20250904 13:59:01.307826 13677 debug-util.cc:398] Leaking SignalData structure 0x7b0800151100 after lost signal to thread 13531
W20250904 13:59:01.309113 13677 debug-util.cc:398] Leaking SignalData structure 0x7b08001276e0 after lost signal to thread 13606
I20250904 13:59:03.315064 10634 tablet_server.cc:178] TabletServer@127.10.98.129:0 shutting down...
I20250904 13:59:03.373865 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:59:03.374832 10634 tablet_replica.cc:331] T 1bf8c99aa65744d79f0e7b0524e57e76 P 69dc5e3b237246b6bfaacc3b5384e10b: stopping tablet replica
I20250904 13:59:03.375712 10634 raft_consensus.cc:2241] T 1bf8c99aa65744d79f0e7b0524e57e76 P 69dc5e3b237246b6bfaacc3b5384e10b [term 3 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:03.376976 10634 raft_consensus.cc:2270] T 1bf8c99aa65744d79f0e7b0524e57e76 P 69dc5e3b237246b6bfaacc3b5384e10b [term 3 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:03.475102 10634 tablet_server.cc:195] TabletServer@127.10.98.129:0 shutdown complete.
I20250904 13:59:03.500726 10634 tablet_server.cc:178] TabletServer@127.10.98.130:0 shutting down...
W20250904 13:59:03.506867 13547 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.98.129:35645: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20250904 13:59:03.514034 13547 consensus_peers.cc:489] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 -> Peer 69dc5e3b237246b6bfaacc3b5384e10b (127.10.98.129:35645): Couldn't send request to peer 69dc5e3b237246b6bfaacc3b5384e10b. Status: Network error: Client connection negotiation failed: client connection to 127.10.98.129:35645: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 13:59:03.535315 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:59:03.536423 10634 tablet_replica.cc:331] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2: stopping tablet replica
I20250904 13:59:03.537293 10634 raft_consensus.cc:2241] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [term 3 LEADER]: Raft consensus shutting down.
I20250904 13:59:03.538944 10634 raft_consensus.cc:2270] T 1bf8c99aa65744d79f0e7b0524e57e76 P 9ab118fc96be4284a051212d6c9e58b2 [term 3 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:03.629769 10634 tablet_server.cc:195] TabletServer@127.10.98.130:0 shutdown complete.
I20250904 13:59:03.651362 10634 tablet_server.cc:178] TabletServer@127.10.98.131:0 shutting down...
I20250904 13:59:03.692732 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:59:03.693527 10634 tablet_replica.cc:331] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792: stopping tablet replica
I20250904 13:59:03.694325 10634 raft_consensus.cc:2241] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 3 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:03.695253 10634 raft_consensus.cc:2270] T 1bf8c99aa65744d79f0e7b0524e57e76 P ccb04f45a1ee4e0e8942995609d0b792 [term 3 FOLLOWER]: Raft consensus is shut down!
W20250904 13:59:04.785116 10634 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 13677)
W20250904 13:59:05.785722 10634 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 13677)
I20250904 13:59:06.326989 10634 tablet_server.cc:195] TabletServer@127.10.98.131:0 shutdown complete.
I20250904 13:59:06.347321 10634 master.cc:561] Master@127.10.98.190:44159 shutting down...
I20250904 13:59:06.380259 10634 raft_consensus.cc:2241] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:06.381047 10634 raft_consensus.cc:2270] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:06.381474 10634 tablet_replica.cc:331] T 00000000000000000000000000000000 P f0a09fc7740b4822bd5175fe9d4f6748: stopping tablet replica
I20250904 13:59:06.402465 10634 master.cc:583] Master@127.10.98.190:44159 shutdown complete.
[ OK ] TServerQuiescingITest.TestQuiesceAndStopTool (30270 ms)
[----------] 8 tests from TServerQuiescingITest (104529 ms total)
[----------] 10 tests from NumReplicas/TServerQuiescingParamITest
[ RUN ] NumReplicas/TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests/0
I20250904 13:59:06.441135 10634 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.98.190:44273
I20250904 13:59:06.442612 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:06.449811 13855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:06.449962 13854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:06.451243 13857 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:06.452152 10634 server_base.cc:1047] running on GCE node
I20250904 13:59:06.453255 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:06.453455 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:06.453573 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994346453559 us; error 0 us; skew 500 ppm
I20250904 13:59:06.454197 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:06.456956 10634 webserver.cc:480] Webserver started at http://127.10.98.190:45615/ using document root <none> and password file <none>
I20250904 13:59:06.457511 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:06.457681 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:06.457937 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:06.459342 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_0.1756994241738102-10634-0/minicluster-data/master-0-root/instance:
uuid: "47ddf6898d7c4422a570c8715dbf043a"
format_stamp: "Formatted at 2025-09-04 13:59:06 on dist-test-slave-jx76"
I20250904 13:59:06.464870 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20250904 13:59:06.469230 13862 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:06.470149 10634 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20250904 13:59:06.470467 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_0.1756994241738102-10634-0/minicluster-data/master-0-root
uuid: "47ddf6898d7c4422a570c8715dbf043a"
format_stamp: "Formatted at 2025-09-04 13:59:06 on dist-test-slave-jx76"
I20250904 13:59:06.470790 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_0.1756994241738102-10634-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_0.1756994241738102-10634-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_0.1756994241738102-10634-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:06.490877 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:06.492316 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:06.537000 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.190:44273
I20250904 13:59:06.537117 13913 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.190:44273 every 8 connection(s)
I20250904 13:59:06.541431 13914 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:06.554350 13914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a: Bootstrap starting.
I20250904 13:59:06.559562 13914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:06.564354 13914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a: No bootstrap required, opened a new log
I20250904 13:59:06.566803 13914 raft_consensus.cc:357] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ddf6898d7c4422a570c8715dbf043a" member_type: VOTER }
I20250904 13:59:06.567276 13914 raft_consensus.cc:383] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:06.567540 13914 raft_consensus.cc:738] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47ddf6898d7c4422a570c8715dbf043a, State: Initialized, Role: FOLLOWER
I20250904 13:59:06.568169 13914 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ddf6898d7c4422a570c8715dbf043a" member_type: VOTER }
I20250904 13:59:06.568667 13914 raft_consensus.cc:397] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:59:06.568886 13914 raft_consensus.cc:491] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250904 13:59:06.569171 13914 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:06.574769 13914 raft_consensus.cc:513] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ddf6898d7c4422a570c8715dbf043a" member_type: VOTER }
I20250904 13:59:06.575356 13914 leader_election.cc:304] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47ddf6898d7c4422a570c8715dbf043a; no voters:
I20250904 13:59:06.576601 13914 leader_election.cc:290] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:59:06.577075 13917 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:06.578517 13917 raft_consensus.cc:695] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a [term 1 LEADER]: Becoming Leader. State: Replica: 47ddf6898d7c4422a570c8715dbf043a, State: Running, Role: LEADER
I20250904 13:59:06.579269 13917 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ddf6898d7c4422a570c8715dbf043a" member_type: VOTER }
I20250904 13:59:06.579900 13914 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a [sys.catalog]: configured and running, proceeding with master startup.
I20250904 13:59:06.584252 13918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47ddf6898d7c4422a570c8715dbf043a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ddf6898d7c4422a570c8715dbf043a" member_type: VOTER } }
I20250904 13:59:06.584540 13919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47ddf6898d7c4422a570c8715dbf043a. Latest consensus state: current_term: 1 leader_uuid: "47ddf6898d7c4422a570c8715dbf043a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47ddf6898d7c4422a570c8715dbf043a" member_type: VOTER } }
I20250904 13:59:06.585182 13918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a [sys.catalog]: This master's current role is: LEADER
I20250904 13:59:06.585449 13919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a [sys.catalog]: This master's current role is: LEADER
I20250904 13:59:06.592931 13924 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 13:59:06.598471 13924 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 13:59:06.599788 10634 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 13:59:06.609899 13924 catalog_manager.cc:1349] Generated new cluster ID: 14f9630842d744fb87fdfd8e7ca51158
I20250904 13:59:06.610260 13924 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 13:59:06.640957 13924 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 13:59:06.642410 13924 catalog_manager.cc:1506] Loading token signing keys...
I20250904 13:59:06.685359 13924 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a: Generated new TSK 0
I20250904 13:59:06.686834 13924 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 13:59:06.732843 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:06.747246 13935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:06.751400 13936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:06.760358 13938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:06.760993 10634 server_base.cc:1047] running on GCE node
I20250904 13:59:06.764266 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:06.764930 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:06.765479 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994346765444 us; error 0 us; skew 500 ppm
I20250904 13:59:06.766868 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:06.769861 10634 webserver.cc:480] Webserver started at http://127.10.98.129:46393/ using document root <none> and password file <none>
I20250904 13:59:06.771389 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:06.772047 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:06.773121 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:06.777345 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_0.1756994241738102-10634-0/minicluster-data/ts-0-root/instance:
uuid: "0f07d61794074240a4de3734d757e81a"
format_stamp: "Formatted at 2025-09-04 13:59:06 on dist-test-slave-jx76"
I20250904 13:59:06.788928 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.011s user 0.009s sys 0.000s
I20250904 13:59:06.798485 13943 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:06.799367 10634 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20250904 13:59:06.799686 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_0.1756994241738102-10634-0/minicluster-data/ts-0-root
uuid: "0f07d61794074240a4de3734d757e81a"
format_stamp: "Formatted at 2025-09-04 13:59:06 on dist-test-slave-jx76"
I20250904 13:59:06.800007 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_0.1756994241738102-10634-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_0.1756994241738102-10634-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_0.1756994241738102-10634-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:06.824339 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:06.825716 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:06.827489 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:59:06.830102 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:59:06.830335 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:06.830596 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:59:06.830771 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:06.879926 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.129:39523
I20250904 13:59:06.880036 14005 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.129:39523 every 8 connection(s)
I20250904 13:59:06.902274 14006 heartbeater.cc:344] Connected to a master server at 127.10.98.190:44273
I20250904 13:59:06.902716 14006 heartbeater.cc:461] Registering TS with master...
I20250904 13:59:06.903585 14006 heartbeater.cc:507] Master 127.10.98.190:44273 requested a full tablet report, sending...
I20250904 13:59:06.905884 13879 ts_manager.cc:194] Registered new tserver with Master: 0f07d61794074240a4de3734d757e81a (127.10.98.129:39523)
I20250904 13:59:06.906270 10634 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.022627574s
I20250904 13:59:06.908239 13879 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51600
I20250904 13:59:06.943428 10634 test_util.cc:276] Using random seed: 535561131
I20250904 13:59:06.968256 13879 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51636:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250904 13:59:07.005065 13971 tablet_service.cc:1468] Processing CreateTablet for tablet 6cc5d033bf8348e5a12de6398efe4c63 (DEFAULT_TABLE table=test-workload [id=17e68530ac67487eac4bd01c6134bd19]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:59:07.006150 13971 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cc5d033bf8348e5a12de6398efe4c63. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:07.022733 14024 tablet_bootstrap.cc:492] T 6cc5d033bf8348e5a12de6398efe4c63 P 0f07d61794074240a4de3734d757e81a: Bootstrap starting.
I20250904 13:59:07.028115 14024 tablet_bootstrap.cc:654] T 6cc5d033bf8348e5a12de6398efe4c63 P 0f07d61794074240a4de3734d757e81a: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:07.033010 14024 tablet_bootstrap.cc:492] T 6cc5d033bf8348e5a12de6398efe4c63 P 0f07d61794074240a4de3734d757e81a: No bootstrap required, opened a new log
I20250904 13:59:07.033456 14024 ts_tablet_manager.cc:1397] T 6cc5d033bf8348e5a12de6398efe4c63 P 0f07d61794074240a4de3734d757e81a: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20250904 13:59:07.035800 14024 raft_consensus.cc:357] T 6cc5d033bf8348e5a12de6398efe4c63 P 0f07d61794074240a4de3734d757e81a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f07d61794074240a4de3734d757e81a" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 39523 } }
I20250904 13:59:07.036273 14024 raft_consensus.cc:738] T 6cc5d033bf8348e5a12de6398efe4c63 P 0f07d61794074240a4de3734d757e81a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f07d61794074240a4de3734d757e81a, State: Initialized, Role: FOLLOWER
I20250904 13:59:07.036765 14024 consensus_queue.cc:260] T 6cc5d033bf8348e5a12de6398efe4c63 P 0f07d61794074240a4de3734d757e81a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0f07d61794074240a4de3734d757e81a" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 39523 } }
I20250904 13:59:07.038908 14006 heartbeater.cc:499] Master 127.10.98.190:44273 was elected leader, sending a full tablet report...
I20250904 13:59:07.039251 14024 ts_tablet_manager.cc:1428] T 6cc5d033bf8348e5a12de6398efe4c63 P 0f07d61794074240a4de3734d757e81a: Time spent starting tablet: real 0.005s user 0.001s sys 0.006s
W20250904 13:59:07.134362 14007 tablet.cc:2378] T 6cc5d033bf8348e5a12de6398efe4c63 P 0f07d61794074240a4de3734d757e81a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250904 13:59:07.565456 13981 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "6cc5d033bf8348e5a12de6398efe4c63"
dest_uuid: "0f07d61794074240a4de3734d757e81a"
from {username='slave'} at 127.0.0.1:49990
I20250904 13:59:08.073374 10634 tablet_server.cc:178] TabletServer@127.10.98.129:0 shutting down...
I20250904 13:59:08.092825 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:59:08.093456 10634 tablet_replica.cc:331] T 6cc5d033bf8348e5a12de6398efe4c63 P 0f07d61794074240a4de3734d757e81a: stopping tablet replica
I20250904 13:59:08.093989 10634 raft_consensus.cc:2241] T 6cc5d033bf8348e5a12de6398efe4c63 P 0f07d61794074240a4de3734d757e81a [term 0 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:08.094444 10634 raft_consensus.cc:2270] T 6cc5d033bf8348e5a12de6398efe4c63 P 0f07d61794074240a4de3734d757e81a [term 0 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:08.113392 10634 tablet_server.cc:195] TabletServer@127.10.98.129:0 shutdown complete.
I20250904 13:59:08.122819 10634 master.cc:561] Master@127.10.98.190:44273 shutting down...
I20250904 13:59:08.139343 10634 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:08.139936 10634 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:08.140242 10634 tablet_replica.cc:331] T 00000000000000000000000000000000 P 47ddf6898d7c4422a570c8715dbf043a: stopping tablet replica
I20250904 13:59:08.159790 10634 master.cc:583] Master@127.10.98.190:44273 shutdown complete.
[ OK ] NumReplicas/TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests/0 (1743 ms)
[ RUN ] NumReplicas/TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests/1
I20250904 13:59:08.184068 10634 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.98.190:35093
I20250904 13:59:08.185184 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:08.190079 14027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:08.191802 14028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:08.191880 14030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:08.192729 10634 server_base.cc:1047] running on GCE node
I20250904 13:59:08.193507 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:08.193718 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:08.193877 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994348193852 us; error 0 us; skew 500 ppm
I20250904 13:59:08.194464 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:08.196875 10634 webserver.cc:480] Webserver started at http://127.10.98.190:36995/ using document root <none> and password file <none>
I20250904 13:59:08.197388 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:08.197579 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:08.197849 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:08.199076 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_1.1756994241738102-10634-0/minicluster-data/master-0-root/instance:
uuid: "a11c581b9a5e4a56930bc1f582cf7c15"
format_stamp: "Formatted at 2025-09-04 13:59:08 on dist-test-slave-jx76"
I20250904 13:59:08.203658 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20250904 13:59:08.207275 14035 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:08.208036 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250904 13:59:08.208313 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_1.1756994241738102-10634-0/minicluster-data/master-0-root
uuid: "a11c581b9a5e4a56930bc1f582cf7c15"
format_stamp: "Formatted at 2025-09-04 13:59:08 on dist-test-slave-jx76"
I20250904 13:59:08.208577 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_1.1756994241738102-10634-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_1.1756994241738102-10634-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_1.1756994241738102-10634-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:08.228628 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:08.229727 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:08.267362 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.190:35093
I20250904 13:59:08.267472 14086 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.190:35093 every 8 connection(s)
I20250904 13:59:08.271579 14087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:08.284001 14087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15: Bootstrap starting.
I20250904 13:59:08.288707 14087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:08.293112 14087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15: No bootstrap required, opened a new log
I20250904 13:59:08.295270 14087 raft_consensus.cc:357] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a11c581b9a5e4a56930bc1f582cf7c15" member_type: VOTER }
I20250904 13:59:08.295679 14087 raft_consensus.cc:383] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:08.295933 14087 raft_consensus.cc:738] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a11c581b9a5e4a56930bc1f582cf7c15, State: Initialized, Role: FOLLOWER
I20250904 13:59:08.296514 14087 consensus_queue.cc:260] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a11c581b9a5e4a56930bc1f582cf7c15" member_type: VOTER }
I20250904 13:59:08.296983 14087 raft_consensus.cc:397] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:59:08.297209 14087 raft_consensus.cc:491] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250904 13:59:08.297480 14087 raft_consensus.cc:3058] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:08.303010 14087 raft_consensus.cc:513] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a11c581b9a5e4a56930bc1f582cf7c15" member_type: VOTER }
I20250904 13:59:08.303582 14087 leader_election.cc:304] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a11c581b9a5e4a56930bc1f582cf7c15; no voters:
I20250904 13:59:08.304903 14087 leader_election.cc:290] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:59:08.305259 14090 raft_consensus.cc:2802] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:08.306685 14090 raft_consensus.cc:695] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15 [term 1 LEADER]: Becoming Leader. State: Replica: a11c581b9a5e4a56930bc1f582cf7c15, State: Running, Role: LEADER
I20250904 13:59:08.307407 14090 consensus_queue.cc:237] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a11c581b9a5e4a56930bc1f582cf7c15" member_type: VOTER }
I20250904 13:59:08.308049 14087 sys_catalog.cc:564] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 13:59:08.310751 14091 sys_catalog.cc:455] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a11c581b9a5e4a56930bc1f582cf7c15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a11c581b9a5e4a56930bc1f582cf7c15" member_type: VOTER } }
I20250904 13:59:08.310844 14092 sys_catalog.cc:455] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a11c581b9a5e4a56930bc1f582cf7c15. Latest consensus state: current_term: 1 leader_uuid: "a11c581b9a5e4a56930bc1f582cf7c15" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a11c581b9a5e4a56930bc1f582cf7c15" member_type: VOTER } }
I20250904 13:59:08.311481 14091 sys_catalog.cc:458] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15 [sys.catalog]: This master's current role is: LEADER
I20250904 13:59:08.311571 14092 sys_catalog.cc:458] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15 [sys.catalog]: This master's current role is: LEADER
I20250904 13:59:08.316159 14095 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 13:59:08.324674 14095 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 13:59:08.326049 10634 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 13:59:08.336627 14095 catalog_manager.cc:1349] Generated new cluster ID: 0ff9ec80adf9428193ae8d68bde8404c
I20250904 13:59:08.336984 14095 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 13:59:08.355037 14095 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 13:59:08.356482 14095 catalog_manager.cc:1506] Loading token signing keys...
I20250904 13:59:08.370268 14095 catalog_manager.cc:5955] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15: Generated new TSK 0
I20250904 13:59:08.370941 14095 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 13:59:08.393677 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:08.399580 14108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:08.400692 14109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:08.402464 14111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:08.402971 10634 server_base.cc:1047] running on GCE node
I20250904 13:59:08.403836 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:08.404035 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:08.404160 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994348404148 us; error 0 us; skew 500 ppm
I20250904 13:59:08.404582 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:08.407048 10634 webserver.cc:480] Webserver started at http://127.10.98.129:40083/ using document root <none> and password file <none>
I20250904 13:59:08.407516 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:08.407676 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:08.407910 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:08.409039 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_1.1756994241738102-10634-0/minicluster-data/ts-0-root/instance:
uuid: "c66bb627c5814e2e8fdc3057848ae139"
format_stamp: "Formatted at 2025-09-04 13:59:08 on dist-test-slave-jx76"
I20250904 13:59:08.414013 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20250904 13:59:08.417672 14116 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:08.418704 10634 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20250904 13:59:08.419047 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_1.1756994241738102-10634-0/minicluster-data/ts-0-root
uuid: "c66bb627c5814e2e8fdc3057848ae139"
format_stamp: "Formatted at 2025-09-04 13:59:08 on dist-test-slave-jx76"
I20250904 13:59:08.419315 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_1.1756994241738102-10634-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_1.1756994241738102-10634-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_1.1756994241738102-10634-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:08.441890 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:08.443053 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:08.444523 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:59:08.446985 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:59:08.447170 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:08.447443 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:59:08.447582 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:08.490998 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.129:33843
I20250904 13:59:08.491094 14178 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.129:33843 every 8 connection(s)
I20250904 13:59:08.496088 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:08.504163 14183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:08.504611 14184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:08.507634 10634 server_base.cc:1047] running on GCE node
W20250904 13:59:08.507999 14186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:08.509577 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:08.509847 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:08.510089 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994348510069 us; error 0 us; skew 500 ppm
I20250904 13:59:08.510809 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:08.510737 14179 heartbeater.cc:344] Connected to a master server at 127.10.98.190:35093
I20250904 13:59:08.511380 14179 heartbeater.cc:461] Registering TS with master...
I20250904 13:59:08.512481 14179 heartbeater.cc:507] Master 127.10.98.190:35093 requested a full tablet report, sending...
I20250904 13:59:08.514086 10634 webserver.cc:480] Webserver started at http://127.10.98.130:33035/ using document root <none> and password file <none>
I20250904 13:59:08.514750 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:08.514953 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:08.515134 14052 ts_manager.cc:194] Registered new tserver with Master: c66bb627c5814e2e8fdc3057848ae139 (127.10.98.129:33843)
I20250904 13:59:08.515265 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:08.516672 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_1.1756994241738102-10634-0/minicluster-data/ts-1-root/instance:
uuid: "136f8ff26b3f418ea0f83d28e4cbd385"
format_stamp: "Formatted at 2025-09-04 13:59:08 on dist-test-slave-jx76"
I20250904 13:59:08.517158 14052 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50566
I20250904 13:59:08.522548 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20250904 13:59:08.526410 14191 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:08.527273 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250904 13:59:08.527577 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_1.1756994241738102-10634-0/minicluster-data/ts-1-root
uuid: "136f8ff26b3f418ea0f83d28e4cbd385"
format_stamp: "Formatted at 2025-09-04 13:59:08 on dist-test-slave-jx76"
I20250904 13:59:08.527899 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_1.1756994241738102-10634-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_1.1756994241738102-10634-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_1.1756994241738102-10634-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:08.541273 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:08.542448 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:08.543943 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:59:08.546335 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:59:08.546558 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:08.546795 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:59:08.546962 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:08.593318 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.130:44817
I20250904 13:59:08.593420 14253 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.130:44817 every 8 connection(s)
I20250904 13:59:08.598734 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:08.607847 14257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:08.611069 14258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:08.615976 14254 heartbeater.cc:344] Connected to a master server at 127.10.98.190:35093
W20250904 13:59:08.616300 14260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:08.616432 14254 heartbeater.cc:461] Registering TS with master...
I20250904 13:59:08.617237 10634 server_base.cc:1047] running on GCE node
I20250904 13:59:08.617394 14254 heartbeater.cc:507] Master 127.10.98.190:35093 requested a full tablet report, sending...
I20250904 13:59:08.618300 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:08.618528 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:08.618749 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994348618729 us; error 0 us; skew 500 ppm
I20250904 13:59:08.619478 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:08.619803 14052 ts_manager.cc:194] Registered new tserver with Master: 136f8ff26b3f418ea0f83d28e4cbd385 (127.10.98.130:44817)
I20250904 13:59:08.621626 14052 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50574
I20250904 13:59:08.622494 10634 webserver.cc:480] Webserver started at http://127.10.98.131:40089/ using document root <none> and password file <none>
I20250904 13:59:08.623057 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:08.623257 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:08.623530 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:08.624894 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_1.1756994241738102-10634-0/minicluster-data/ts-2-root/instance:
uuid: "cb3715dbf26a4c5c9c6d28d452b1ca99"
format_stamp: "Formatted at 2025-09-04 13:59:08 on dist-test-slave-jx76"
I20250904 13:59:08.629911 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20250904 13:59:08.633699 14265 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:08.634658 10634 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20250904 13:59:08.634949 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_1.1756994241738102-10634-0/minicluster-data/ts-2-root
uuid: "cb3715dbf26a4c5c9c6d28d452b1ca99"
format_stamp: "Formatted at 2025-09-04 13:59:08 on dist-test-slave-jx76"
I20250904 13:59:08.635213 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_1.1756994241738102-10634-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_1.1756994241738102-10634-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests_1.1756994241738102-10634-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:08.645774 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:08.647136 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:08.648774 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:59:08.651290 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:59:08.651502 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:08.651743 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:59:08.651903 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:08.698604 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.131:40301
I20250904 13:59:08.698724 14327 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.131:40301 every 8 connection(s)
I20250904 13:59:08.713202 14328 heartbeater.cc:344] Connected to a master server at 127.10.98.190:35093
I20250904 13:59:08.713614 14328 heartbeater.cc:461] Registering TS with master...
I20250904 13:59:08.714439 14328 heartbeater.cc:507] Master 127.10.98.190:35093 requested a full tablet report, sending...
I20250904 13:59:08.716429 14052 ts_manager.cc:194] Registered new tserver with Master: cb3715dbf26a4c5c9c6d28d452b1ca99 (127.10.98.131:40301)
I20250904 13:59:08.716806 10634 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014726204s
I20250904 13:59:08.718051 14052 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50586
I20250904 13:59:08.751787 10634 test_util.cc:276] Using random seed: 537369489
I20250904 13:59:08.776064 14052 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50614:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 13:59:08.777941 14052 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 13:59:08.819720 14219 tablet_service.cc:1468] Processing CreateTablet for tablet a0a95538505b4af5958e0eafe03d02c5 (DEFAULT_TABLE table=test-workload [id=a1a38b3f824a406db49a91f8f94e01c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:59:08.821273 14219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0a95538505b4af5958e0eafe03d02c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:08.823078 14293 tablet_service.cc:1468] Processing CreateTablet for tablet a0a95538505b4af5958e0eafe03d02c5 (DEFAULT_TABLE table=test-workload [id=a1a38b3f824a406db49a91f8f94e01c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:59:08.824549 14293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0a95538505b4af5958e0eafe03d02c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:08.826323 14144 tablet_service.cc:1468] Processing CreateTablet for tablet a0a95538505b4af5958e0eafe03d02c5 (DEFAULT_TABLE table=test-workload [id=a1a38b3f824a406db49a91f8f94e01c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:59:08.827862 14144 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0a95538505b4af5958e0eafe03d02c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:08.839309 14349 tablet_bootstrap.cc:492] T a0a95538505b4af5958e0eafe03d02c5 P 136f8ff26b3f418ea0f83d28e4cbd385: Bootstrap starting.
I20250904 13:59:08.845443 14350 tablet_bootstrap.cc:492] T a0a95538505b4af5958e0eafe03d02c5 P cb3715dbf26a4c5c9c6d28d452b1ca99: Bootstrap starting.
I20250904 13:59:08.846588 14349 tablet_bootstrap.cc:654] T a0a95538505b4af5958e0eafe03d02c5 P 136f8ff26b3f418ea0f83d28e4cbd385: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:08.848496 14351 tablet_bootstrap.cc:492] T a0a95538505b4af5958e0eafe03d02c5 P c66bb627c5814e2e8fdc3057848ae139: Bootstrap starting.
I20250904 13:59:08.853137 14350 tablet_bootstrap.cc:654] T a0a95538505b4af5958e0eafe03d02c5 P cb3715dbf26a4c5c9c6d28d452b1ca99: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:08.853565 14349 tablet_bootstrap.cc:492] T a0a95538505b4af5958e0eafe03d02c5 P 136f8ff26b3f418ea0f83d28e4cbd385: No bootstrap required, opened a new log
I20250904 13:59:08.854202 14349 ts_tablet_manager.cc:1397] T a0a95538505b4af5958e0eafe03d02c5 P 136f8ff26b3f418ea0f83d28e4cbd385: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.006s
I20250904 13:59:08.855863 14351 tablet_bootstrap.cc:654] T a0a95538505b4af5958e0eafe03d02c5 P c66bb627c5814e2e8fdc3057848ae139: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:08.857486 14349 raft_consensus.cc:357] T a0a95538505b4af5958e0eafe03d02c5 P 136f8ff26b3f418ea0f83d28e4cbd385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "136f8ff26b3f418ea0f83d28e4cbd385" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 44817 } } peers { permanent_uuid: "cb3715dbf26a4c5c9c6d28d452b1ca99" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40301 } } peers { permanent_uuid: "c66bb627c5814e2e8fdc3057848ae139" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 33843 } }
I20250904 13:59:08.858399 14349 raft_consensus.cc:738] T a0a95538505b4af5958e0eafe03d02c5 P 136f8ff26b3f418ea0f83d28e4cbd385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 136f8ff26b3f418ea0f83d28e4cbd385, State: Initialized, Role: FOLLOWER
I20250904 13:59:08.859742 14350 tablet_bootstrap.cc:492] T a0a95538505b4af5958e0eafe03d02c5 P cb3715dbf26a4c5c9c6d28d452b1ca99: No bootstrap required, opened a new log
I20250904 13:59:08.859113 14349 consensus_queue.cc:260] T a0a95538505b4af5958e0eafe03d02c5 P 136f8ff26b3f418ea0f83d28e4cbd385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "136f8ff26b3f418ea0f83d28e4cbd385" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 44817 } } peers { permanent_uuid: "cb3715dbf26a4c5c9c6d28d452b1ca99" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40301 } } peers { permanent_uuid: "c66bb627c5814e2e8fdc3057848ae139" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 33843 } }
I20250904 13:59:08.860345 14350 ts_tablet_manager.cc:1397] T a0a95538505b4af5958e0eafe03d02c5 P cb3715dbf26a4c5c9c6d28d452b1ca99: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.004s
I20250904 13:59:08.862161 14254 heartbeater.cc:499] Master 127.10.98.190:35093 was elected leader, sending a full tablet report...
I20250904 13:59:08.862840 14349 ts_tablet_manager.cc:1428] T a0a95538505b4af5958e0eafe03d02c5 P 136f8ff26b3f418ea0f83d28e4cbd385: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20250904 13:59:08.863410 14351 tablet_bootstrap.cc:492] T a0a95538505b4af5958e0eafe03d02c5 P c66bb627c5814e2e8fdc3057848ae139: No bootstrap required, opened a new log
I20250904 13:59:08.863478 14350 raft_consensus.cc:357] T a0a95538505b4af5958e0eafe03d02c5 P cb3715dbf26a4c5c9c6d28d452b1ca99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "136f8ff26b3f418ea0f83d28e4cbd385" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 44817 } } peers { permanent_uuid: "cb3715dbf26a4c5c9c6d28d452b1ca99" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40301 } } peers { permanent_uuid: "c66bb627c5814e2e8fdc3057848ae139" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 33843 } }
I20250904 13:59:08.864065 14351 ts_tablet_manager.cc:1397] T a0a95538505b4af5958e0eafe03d02c5 P c66bb627c5814e2e8fdc3057848ae139: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.009s
I20250904 13:59:08.864388 14350 raft_consensus.cc:738] T a0a95538505b4af5958e0eafe03d02c5 P cb3715dbf26a4c5c9c6d28d452b1ca99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb3715dbf26a4c5c9c6d28d452b1ca99, State: Initialized, Role: FOLLOWER
I20250904 13:59:08.865192 14350 consensus_queue.cc:260] T a0a95538505b4af5958e0eafe03d02c5 P cb3715dbf26a4c5c9c6d28d452b1ca99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "136f8ff26b3f418ea0f83d28e4cbd385" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 44817 } } peers { permanent_uuid: "cb3715dbf26a4c5c9c6d28d452b1ca99" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40301 } } peers { permanent_uuid: "c66bb627c5814e2e8fdc3057848ae139" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 33843 } }
I20250904 13:59:08.867879 14351 raft_consensus.cc:357] T a0a95538505b4af5958e0eafe03d02c5 P c66bb627c5814e2e8fdc3057848ae139 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "136f8ff26b3f418ea0f83d28e4cbd385" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 44817 } } peers { permanent_uuid: "cb3715dbf26a4c5c9c6d28d452b1ca99" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40301 } } peers { permanent_uuid: "c66bb627c5814e2e8fdc3057848ae139" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 33843 } }
I20250904 13:59:08.869153 14351 raft_consensus.cc:738] T a0a95538505b4af5958e0eafe03d02c5 P c66bb627c5814e2e8fdc3057848ae139 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c66bb627c5814e2e8fdc3057848ae139, State: Initialized, Role: FOLLOWER
I20250904 13:59:08.870079 14351 consensus_queue.cc:260] T a0a95538505b4af5958e0eafe03d02c5 P c66bb627c5814e2e8fdc3057848ae139 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "136f8ff26b3f418ea0f83d28e4cbd385" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 44817 } } peers { permanent_uuid: "cb3715dbf26a4c5c9c6d28d452b1ca99" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 40301 } } peers { permanent_uuid: "c66bb627c5814e2e8fdc3057848ae139" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 33843 } }
I20250904 13:59:08.871937 14328 heartbeater.cc:499] Master 127.10.98.190:35093 was elected leader, sending a full tablet report...
I20250904 13:59:08.872871 14350 ts_tablet_manager.cc:1428] T a0a95538505b4af5958e0eafe03d02c5 P cb3715dbf26a4c5c9c6d28d452b1ca99: Time spent starting tablet: real 0.012s user 0.009s sys 0.005s
I20250904 13:59:08.876534 14179 heartbeater.cc:499] Master 127.10.98.190:35093 was elected leader, sending a full tablet report...
I20250904 13:59:08.877303 14351 ts_tablet_manager.cc:1428] T a0a95538505b4af5958e0eafe03d02c5 P c66bb627c5814e2e8fdc3057848ae139: Time spent starting tablet: real 0.013s user 0.007s sys 0.002s
I20250904 13:59:09.385488 14154 tablet_service.cc:1940] Received Run Leader Election RPC: tablet_id: "a0a95538505b4af5958e0eafe03d02c5"
dest_uuid: "c66bb627c5814e2e8fdc3057848ae139"
from {username='slave'} at 127.0.0.1:50466
I20250904 13:59:09.887218 10634 tablet_server.cc:178] TabletServer@127.10.98.129:0 shutting down...
I20250904 13:59:09.911800 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:59:09.912446 10634 tablet_replica.cc:331] T a0a95538505b4af5958e0eafe03d02c5 P c66bb627c5814e2e8fdc3057848ae139: stopping tablet replica
I20250904 13:59:09.912969 10634 raft_consensus.cc:2241] T a0a95538505b4af5958e0eafe03d02c5 P c66bb627c5814e2e8fdc3057848ae139 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:09.913368 10634 raft_consensus.cc:2270] T a0a95538505b4af5958e0eafe03d02c5 P c66bb627c5814e2e8fdc3057848ae139 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:09.932288 10634 tablet_server.cc:195] TabletServer@127.10.98.129:0 shutdown complete.
I20250904 13:59:09.941756 10634 tablet_server.cc:178] TabletServer@127.10.98.130:0 shutting down...
I20250904 13:59:09.961699 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:59:09.962347 10634 tablet_replica.cc:331] T a0a95538505b4af5958e0eafe03d02c5 P 136f8ff26b3f418ea0f83d28e4cbd385: stopping tablet replica
I20250904 13:59:09.962872 10634 raft_consensus.cc:2241] T a0a95538505b4af5958e0eafe03d02c5 P 136f8ff26b3f418ea0f83d28e4cbd385 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:09.963267 10634 raft_consensus.cc:2270] T a0a95538505b4af5958e0eafe03d02c5 P 136f8ff26b3f418ea0f83d28e4cbd385 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:09.981935 10634 tablet_server.cc:195] TabletServer@127.10.98.130:0 shutdown complete.
I20250904 13:59:09.991439 10634 tablet_server.cc:178] TabletServer@127.10.98.131:0 shutting down...
I20250904 13:59:10.009300 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:59:10.009923 10634 tablet_replica.cc:331] T a0a95538505b4af5958e0eafe03d02c5 P cb3715dbf26a4c5c9c6d28d452b1ca99: stopping tablet replica
I20250904 13:59:10.010505 10634 raft_consensus.cc:2241] T a0a95538505b4af5958e0eafe03d02c5 P cb3715dbf26a4c5c9c6d28d452b1ca99 [term 0 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:10.010931 10634 raft_consensus.cc:2270] T a0a95538505b4af5958e0eafe03d02c5 P cb3715dbf26a4c5c9c6d28d452b1ca99 [term 0 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:10.029793 10634 tablet_server.cc:195] TabletServer@127.10.98.131:0 shutdown complete.
I20250904 13:59:10.038982 10634 master.cc:561] Master@127.10.98.190:35093 shutting down...
I20250904 13:59:10.056313 10634 raft_consensus.cc:2241] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:10.056891 10634 raft_consensus.cc:2270] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:10.057183 10634 tablet_replica.cc:331] T 00000000000000000000000000000000 P a11c581b9a5e4a56930bc1f582cf7c15: stopping tablet replica
I20250904 13:59:10.075565 10634 master.cc:583] Master@127.10.98.190:35093 shutdown complete.
[ OK ] NumReplicas/TServerQuiescingParamITest.TestQuiescingServerRejectsElectionRequests/1 (1918 ms)
[ RUN ] NumReplicas/TServerQuiescingParamITest.TestNoElectionsForNewReplicas/0
I20250904 13:59:10.102816 10634 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.98.190:38271
I20250904 13:59:10.104029 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:10.109257 14358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:10.109491 14359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:10.110715 14361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:10.112861 10634 server_base.cc:1047] running on GCE node
I20250904 13:59:10.113626 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:10.113801 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:10.113963 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994350113942 us; error 0 us; skew 500 ppm
I20250904 13:59:10.114502 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:10.116925 10634 webserver.cc:480] Webserver started at http://127.10.98.190:42353/ using document root <none> and password file <none>
I20250904 13:59:10.117458 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:10.117653 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:10.117930 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:10.119107 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_0.1756994241738102-10634-0/minicluster-data/master-0-root/instance:
uuid: "fb223b9a51754aac9c57648a6d1efa20"
format_stamp: "Formatted at 2025-09-04 13:59:10 on dist-test-slave-jx76"
I20250904 13:59:10.123935 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250904 13:59:10.127727 14366 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:10.128508 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20250904 13:59:10.128829 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_0.1756994241738102-10634-0/minicluster-data/master-0-root
uuid: "fb223b9a51754aac9c57648a6d1efa20"
format_stamp: "Formatted at 2025-09-04 13:59:10 on dist-test-slave-jx76"
I20250904 13:59:10.129119 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_0.1756994241738102-10634-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_0.1756994241738102-10634-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_0.1756994241738102-10634-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:10.148445 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:10.149546 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:10.187862 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.190:38271
I20250904 13:59:10.187970 14417 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.190:38271 every 8 connection(s)
I20250904 13:59:10.191958 14418 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:10.204223 14418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20: Bootstrap starting.
I20250904 13:59:10.208807 14418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:10.213210 14418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20: No bootstrap required, opened a new log
I20250904 13:59:10.215306 14418 raft_consensus.cc:357] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb223b9a51754aac9c57648a6d1efa20" member_type: VOTER }
I20250904 13:59:10.215726 14418 raft_consensus.cc:383] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:10.215900 14418 raft_consensus.cc:738] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb223b9a51754aac9c57648a6d1efa20, State: Initialized, Role: FOLLOWER
I20250904 13:59:10.216473 14418 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb223b9a51754aac9c57648a6d1efa20" member_type: VOTER }
I20250904 13:59:10.216941 14418 raft_consensus.cc:397] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:59:10.217130 14418 raft_consensus.cc:491] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250904 13:59:10.217341 14418 raft_consensus.cc:3058] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:10.222595 14418 raft_consensus.cc:513] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb223b9a51754aac9c57648a6d1efa20" member_type: VOTER }
I20250904 13:59:10.223148 14418 leader_election.cc:304] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb223b9a51754aac9c57648a6d1efa20; no voters:
I20250904 13:59:10.224335 14418 leader_election.cc:290] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:59:10.224712 14421 raft_consensus.cc:2802] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:10.226203 14421 raft_consensus.cc:695] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20 [term 1 LEADER]: Becoming Leader. State: Replica: fb223b9a51754aac9c57648a6d1efa20, State: Running, Role: LEADER
I20250904 13:59:10.226953 14421 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb223b9a51754aac9c57648a6d1efa20" member_type: VOTER }
I20250904 13:59:10.227672 14418 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 13:59:10.229884 14423 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb223b9a51754aac9c57648a6d1efa20. Latest consensus state: current_term: 1 leader_uuid: "fb223b9a51754aac9c57648a6d1efa20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb223b9a51754aac9c57648a6d1efa20" member_type: VOTER } }
I20250904 13:59:10.230621 14423 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20 [sys.catalog]: This master's current role is: LEADER
I20250904 13:59:10.229884 14422 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb223b9a51754aac9c57648a6d1efa20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb223b9a51754aac9c57648a6d1efa20" member_type: VOTER } }
I20250904 13:59:10.231634 14422 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20 [sys.catalog]: This master's current role is: LEADER
I20250904 13:59:10.233637 14426 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 13:59:10.239533 14426 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 13:59:10.244513 10634 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 13:59:10.249301 14426 catalog_manager.cc:1349] Generated new cluster ID: 54b8b5b1985344f385ccc1295ed15af3
I20250904 13:59:10.249619 14426 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 13:59:10.279889 14426 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 13:59:10.281314 14426 catalog_manager.cc:1506] Loading token signing keys...
I20250904 13:59:10.296315 14426 catalog_manager.cc:5955] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20: Generated new TSK 0
I20250904 13:59:10.297012 14426 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 13:59:10.314766 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:10.320864 14439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:10.322022 14440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:10.323778 14442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:10.324507 10634 server_base.cc:1047] running on GCE node
I20250904 13:59:10.325321 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:10.325510 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:10.325641 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994350325629 us; error 0 us; skew 500 ppm
I20250904 13:59:10.326102 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:10.328572 10634 webserver.cc:480] Webserver started at http://127.10.98.129:44907/ using document root <none> and password file <none>
I20250904 13:59:10.329056 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:10.329213 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:10.329481 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:10.330789 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_0.1756994241738102-10634-0/minicluster-data/ts-0-root/instance:
uuid: "81411e2807144c1aba3981c03d6c69a4"
format_stamp: "Formatted at 2025-09-04 13:59:10 on dist-test-slave-jx76"
I20250904 13:59:10.336124 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20250904 13:59:10.339916 14447 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:10.340814 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20250904 13:59:10.341109 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_0.1756994241738102-10634-0/minicluster-data/ts-0-root
uuid: "81411e2807144c1aba3981c03d6c69a4"
format_stamp: "Formatted at 2025-09-04 13:59:10 on dist-test-slave-jx76"
I20250904 13:59:10.341385 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_0.1756994241738102-10634-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_0.1756994241738102-10634-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_0.1756994241738102-10634-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:10.351699 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:10.352905 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:10.354430 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:59:10.356880 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:59:10.357081 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:10.357321 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:59:10.357482 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:10.401046 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.129:43537
I20250904 13:59:10.401149 14509 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.129:43537 every 8 connection(s)
I20250904 13:59:10.417038 14510 heartbeater.cc:344] Connected to a master server at 127.10.98.190:38271
I20250904 13:59:10.417435 14510 heartbeater.cc:461] Registering TS with master...
I20250904 13:59:10.418283 14510 heartbeater.cc:507] Master 127.10.98.190:38271 requested a full tablet report, sending...
I20250904 13:59:10.420740 14383 ts_manager.cc:194] Registered new tserver with Master: 81411e2807144c1aba3981c03d6c69a4 (127.10.98.129:43537)
I20250904 13:59:10.421672 10634 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.016782048s
I20250904 13:59:10.422780 14383 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49868
I20250904 13:59:10.455857 10634 test_util.cc:276] Using random seed: 539073559
I20250904 13:59:10.479723 14383 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49908:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250904 13:59:10.531889 14475 tablet_service.cc:1468] Processing CreateTablet for tablet 4b92eab78a464ac7be3d33e4da63d488 (DEFAULT_TABLE table=test-workload [id=0e9dde184b1142fda0f442cae585dd1d]), partition=RANGE (key) PARTITION VALUES < 214748364
I20250904 13:59:10.532130 14474 tablet_service.cc:1468] Processing CreateTablet for tablet 6c821980dec14c0a968c6f55ace0fc3f (DEFAULT_TABLE table=test-workload [id=0e9dde184b1142fda0f442cae585dd1d]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20250904 13:59:10.533504 14475 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4b92eab78a464ac7be3d33e4da63d488. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:10.534152 14470 tablet_service.cc:1468] Processing CreateTablet for tablet 5c1b17ca1fa345ca8c59f939422bb44f (DEFAULT_TABLE table=test-workload [id=0e9dde184b1142fda0f442cae585dd1d]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20250904 13:59:10.534579 14469 tablet_service.cc:1468] Processing CreateTablet for tablet 2ffc529566c94e2e929beebb254c5ea1 (DEFAULT_TABLE table=test-workload [id=0e9dde184b1142fda0f442cae585dd1d]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20250904 13:59:10.535552 14470 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c1b17ca1fa345ca8c59f939422bb44f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:10.535230 14472 tablet_service.cc:1468] Processing CreateTablet for tablet e0bf641b345443a4a37326c8c5c346ff (DEFAULT_TABLE table=test-workload [id=0e9dde184b1142fda0f442cae585dd1d]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20250904 13:59:10.536556 14472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e0bf641b345443a4a37326c8c5c346ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:10.536886 14468 tablet_service.cc:1468] Processing CreateTablet for tablet ccd9f92e96034550a059b0240c4abd44 (DEFAULT_TABLE table=test-workload [id=0e9dde184b1142fda0f442cae585dd1d]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20250904 13:59:10.538188 14468 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccd9f92e96034550a059b0240c4abd44. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:10.539328 14471 tablet_service.cc:1468] Processing CreateTablet for tablet ba40b00cc1614ca1aedbf6852db62c27 (DEFAULT_TABLE table=test-workload [id=0e9dde184b1142fda0f442cae585dd1d]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20250904 13:59:10.540715 14471 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba40b00cc1614ca1aedbf6852db62c27. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:10.542513 14469 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2ffc529566c94e2e929beebb254c5ea1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:10.542783 14473 tablet_service.cc:1468] Processing CreateTablet for tablet 73d3f377465c4519a822c65abe9cdd36 (DEFAULT_TABLE table=test-workload [id=0e9dde184b1142fda0f442cae585dd1d]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20250904 13:59:10.544308 14473 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 73d3f377465c4519a822c65abe9cdd36. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:10.547797 14474 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c821980dec14c0a968c6f55ace0fc3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:10.552138 14467 tablet_service.cc:1468] Processing CreateTablet for tablet 7a621adfa43047bfa1feb039ded81dac (DEFAULT_TABLE table=test-workload [id=0e9dde184b1142fda0f442cae585dd1d]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20250904 13:59:10.553712 14467 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7a621adfa43047bfa1feb039ded81dac. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:10.560438 14466 tablet_service.cc:1468] Processing CreateTablet for tablet c0e5555020ff4f06a76e21ba21285500 (DEFAULT_TABLE table=test-workload [id=0e9dde184b1142fda0f442cae585dd1d]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20250904 13:59:10.562480 14466 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0e5555020ff4f06a76e21ba21285500. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:10.599407 14528 tablet_bootstrap.cc:492] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4: Bootstrap starting.
I20250904 13:59:10.606318 14528 tablet_bootstrap.cc:654] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:10.615717 14528 tablet_bootstrap.cc:492] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4: No bootstrap required, opened a new log
I20250904 13:59:10.616103 14528 ts_tablet_manager.cc:1397] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4: Time spent bootstrapping tablet: real 0.017s user 0.014s sys 0.000s
I20250904 13:59:10.617965 14528 raft_consensus.cc:357] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:10.618445 14528 raft_consensus.cc:383] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:10.618642 14528 raft_consensus.cc:738] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81411e2807144c1aba3981c03d6c69a4, State: Initialized, Role: FOLLOWER
I20250904 13:59:10.619154 14528 consensus_queue.cc:260] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:10.619622 14528 raft_consensus.cc:397] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
W20250904 13:59:10.619882 14528 raft_consensus.cc:399] Couldn't start leader election: Illegal state: leader elections are disabled
I20250904 13:59:10.621567 14510 heartbeater.cc:499] Master 127.10.98.190:38271 was elected leader, sending a full tablet report...
I20250904 13:59:10.621933 14528 ts_tablet_manager.cc:1428] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250904 13:59:10.622819 14528 tablet_bootstrap.cc:492] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4: Bootstrap starting.
I20250904 13:59:10.627728 14528 tablet_bootstrap.cc:654] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:10.632294 14528 tablet_bootstrap.cc:492] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4: No bootstrap required, opened a new log
I20250904 13:59:10.632764 14528 ts_tablet_manager.cc:1397] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20250904 13:59:10.635301 14528 raft_consensus.cc:357] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:10.635900 14528 raft_consensus.cc:383] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:10.636128 14528 raft_consensus.cc:738] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81411e2807144c1aba3981c03d6c69a4, State: Initialized, Role: FOLLOWER
I20250904 13:59:10.636646 14528 consensus_queue.cc:260] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:10.637104 14528 raft_consensus.cc:397] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:59:10.638916 14528 ts_tablet_manager.cc:1428] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250904 13:59:10.639650 14528 tablet_bootstrap.cc:492] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4: Bootstrap starting.
I20250904 13:59:10.644342 14528 tablet_bootstrap.cc:654] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:10.648865 14528 tablet_bootstrap.cc:492] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4: No bootstrap required, opened a new log
I20250904 13:59:10.649385 14528 ts_tablet_manager.cc:1397] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20250904 13:59:10.651629 14528 raft_consensus.cc:357] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:10.652205 14528 raft_consensus.cc:383] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:10.652534 14528 raft_consensus.cc:738] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81411e2807144c1aba3981c03d6c69a4, State: Initialized, Role: FOLLOWER
I20250904 13:59:10.653177 14528 consensus_queue.cc:260] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:10.653744 14528 raft_consensus.cc:397] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:59:10.655580 14528 ts_tablet_manager.cc:1428] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20250904 13:59:10.656589 14528 tablet_bootstrap.cc:492] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4: Bootstrap starting.
I20250904 13:59:10.661188 14528 tablet_bootstrap.cc:654] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:10.665746 14528 tablet_bootstrap.cc:492] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4: No bootstrap required, opened a new log
I20250904 13:59:10.666188 14528 ts_tablet_manager.cc:1397] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20250904 13:59:10.668565 14528 raft_consensus.cc:357] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:10.669039 14528 raft_consensus.cc:383] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:10.669256 14528 raft_consensus.cc:738] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81411e2807144c1aba3981c03d6c69a4, State: Initialized, Role: FOLLOWER
I20250904 13:59:10.669790 14528 consensus_queue.cc:260] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:10.670348 14528 raft_consensus.cc:397] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:59:10.671854 14528 ts_tablet_manager.cc:1428] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250904 13:59:10.672654 14528 tablet_bootstrap.cc:492] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4: Bootstrap starting.
I20250904 13:59:10.678247 14528 tablet_bootstrap.cc:654] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:10.683182 14528 tablet_bootstrap.cc:492] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4: No bootstrap required, opened a new log
I20250904 13:59:10.683702 14528 ts_tablet_manager.cc:1397] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20250904 13:59:10.686098 14528 raft_consensus.cc:357] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:10.686573 14528 raft_consensus.cc:383] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:10.686811 14528 raft_consensus.cc:738] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81411e2807144c1aba3981c03d6c69a4, State: Initialized, Role: FOLLOWER
I20250904 13:59:10.687431 14528 consensus_queue.cc:260] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:10.687952 14528 raft_consensus.cc:397] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:59:10.689445 14528 ts_tablet_manager.cc:1428] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20250904 13:59:10.690325 14528 tablet_bootstrap.cc:492] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4: Bootstrap starting.
I20250904 13:59:10.694667 14528 tablet_bootstrap.cc:654] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:10.699358 14528 tablet_bootstrap.cc:492] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4: No bootstrap required, opened a new log
I20250904 13:59:10.699841 14528 ts_tablet_manager.cc:1397] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.001s
I20250904 13:59:10.702270 14528 raft_consensus.cc:357] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:10.702693 14528 raft_consensus.cc:383] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:10.702883 14528 raft_consensus.cc:738] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81411e2807144c1aba3981c03d6c69a4, State: Initialized, Role: FOLLOWER
I20250904 13:59:10.703409 14528 consensus_queue.cc:260] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:10.703862 14528 raft_consensus.cc:397] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:59:10.705320 14528 ts_tablet_manager.cc:1428] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250904 13:59:10.706221 14528 tablet_bootstrap.cc:492] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4: Bootstrap starting.
I20250904 13:59:10.710836 14528 tablet_bootstrap.cc:654] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:10.716348 14528 tablet_bootstrap.cc:492] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4: No bootstrap required, opened a new log
I20250904 13:59:10.716849 14528 ts_tablet_manager.cc:1397] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20250904 13:59:10.718794 14528 raft_consensus.cc:357] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:10.719369 14528 raft_consensus.cc:383] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:10.719592 14528 raft_consensus.cc:738] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81411e2807144c1aba3981c03d6c69a4, State: Initialized, Role: FOLLOWER
I20250904 13:59:10.720265 14528 consensus_queue.cc:260] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:10.720939 14528 raft_consensus.cc:397] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:59:10.723019 14528 ts_tablet_manager.cc:1428] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20250904 13:59:10.723882 14528 tablet_bootstrap.cc:492] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4: Bootstrap starting.
I20250904 13:59:10.729490 14528 tablet_bootstrap.cc:654] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:10.734344 14528 tablet_bootstrap.cc:492] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4: No bootstrap required, opened a new log
I20250904 13:59:10.734812 14528 ts_tablet_manager.cc:1397] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20250904 13:59:10.736829 14528 raft_consensus.cc:357] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:10.737234 14528 raft_consensus.cc:383] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:10.737442 14528 raft_consensus.cc:738] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81411e2807144c1aba3981c03d6c69a4, State: Initialized, Role: FOLLOWER
I20250904 13:59:10.737975 14528 consensus_queue.cc:260] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:10.738477 14528 raft_consensus.cc:397] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:59:10.739624 14528 ts_tablet_manager.cc:1428] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250904 13:59:10.740497 14528 tablet_bootstrap.cc:492] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4: Bootstrap starting.
I20250904 13:59:10.744958 14528 tablet_bootstrap.cc:654] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:10.749274 14528 tablet_bootstrap.cc:492] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4: No bootstrap required, opened a new log
I20250904 13:59:10.749706 14528 ts_tablet_manager.cc:1397] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20250904 13:59:10.751561 14528 raft_consensus.cc:357] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:10.751973 14528 raft_consensus.cc:383] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:10.752171 14528 raft_consensus.cc:738] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81411e2807144c1aba3981c03d6c69a4, State: Initialized, Role: FOLLOWER
I20250904 13:59:10.752661 14528 consensus_queue.cc:260] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:10.753252 14528 raft_consensus.cc:397] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:59:10.754513 14528 ts_tablet_manager.cc:1428] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20250904 13:59:10.755230 14528 tablet_bootstrap.cc:492] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4: Bootstrap starting.
I20250904 13:59:10.759461 14528 tablet_bootstrap.cc:654] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:10.763844 14528 tablet_bootstrap.cc:492] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4: No bootstrap required, opened a new log
I20250904 13:59:10.764326 14528 ts_tablet_manager.cc:1397] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20250904 13:59:10.766307 14528 raft_consensus.cc:357] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:10.766717 14528 raft_consensus.cc:383] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:10.766929 14528 raft_consensus.cc:738] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81411e2807144c1aba3981c03d6c69a4, State: Initialized, Role: FOLLOWER
I20250904 13:59:10.767467 14528 consensus_queue.cc:260] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:10.768091 14528 raft_consensus.cc:397] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:59:10.769541 14528 ts_tablet_manager.cc:1428] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20250904 13:59:10.838083 14530 raft_consensus.cc:668] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4: failed to trigger leader election: Illegal state: leader elections are disabled
I20250904 13:59:12.131731 14540 raft_consensus.cc:491] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:12.132166 14540 raft_consensus.cc:513] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.132758 14540 leader_election.cc:304] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81411e2807144c1aba3981c03d6c69a4; no voters:
I20250904 13:59:12.133791 14540 leader_election.cc:290] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20250904 13:59:12.134102 14541 raft_consensus.cc:2802] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:12.134487 14541 raft_consensus.cc:491] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:12.134739 14541 raft_consensus.cc:3058] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:12.140375 14541 raft_consensus.cc:513] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.141014 14541 leader_election.cc:304] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81411e2807144c1aba3981c03d6c69a4; no voters:
I20250904 13:59:12.141534 14541 leader_election.cc:290] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:59:12.141693 14540 raft_consensus.cc:2802] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:12.142190 14540 raft_consensus.cc:695] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 [term 1 LEADER]: Becoming Leader. State: Replica: 81411e2807144c1aba3981c03d6c69a4, State: Running, Role: LEADER
I20250904 13:59:12.142851 14540 consensus_queue.cc:237] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.150461 14383 catalog_manager.cc:5582] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81411e2807144c1aba3981c03d6c69a4 (127.10.98.129). New cstate: current_term: 1 leader_uuid: "81411e2807144c1aba3981c03d6c69a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } health_report { overall_health: HEALTHY } } }
I20250904 13:59:12.340606 14540 raft_consensus.cc:491] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:12.341208 14540 raft_consensus.cc:513] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.341869 14540 leader_election.cc:304] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81411e2807144c1aba3981c03d6c69a4; no voters:
I20250904 13:59:12.342443 14540 leader_election.cc:290] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20250904 13:59:12.342664 14541 raft_consensus.cc:2802] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:12.343026 14541 raft_consensus.cc:491] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:12.343252 14541 raft_consensus.cc:3058] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:12.348718 14541 raft_consensus.cc:513] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.349357 14541 leader_election.cc:304] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81411e2807144c1aba3981c03d6c69a4; no voters:
I20250904 13:59:12.349870 14541 leader_election.cc:290] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:59:12.350044 14540 raft_consensus.cc:2802] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:12.350548 14540 raft_consensus.cc:695] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 [term 1 LEADER]: Becoming Leader. State: Replica: 81411e2807144c1aba3981c03d6c69a4, State: Running, Role: LEADER
I20250904 13:59:12.351320 14540 consensus_queue.cc:237] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.357506 14383 catalog_manager.cc:5582] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81411e2807144c1aba3981c03d6c69a4 (127.10.98.129). New cstate: current_term: 1 leader_uuid: "81411e2807144c1aba3981c03d6c69a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } health_report { overall_health: HEALTHY } } }
I20250904 13:59:12.431186 14540 raft_consensus.cc:491] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:12.431655 14540 raft_consensus.cc:513] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.432229 14540 leader_election.cc:304] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81411e2807144c1aba3981c03d6c69a4; no voters:
I20250904 13:59:12.432698 14540 leader_election.cc:290] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20250904 13:59:12.432848 14541 raft_consensus.cc:2802] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:12.433209 14541 raft_consensus.cc:491] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:12.433458 14541 raft_consensus.cc:3058] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:12.438972 14541 raft_consensus.cc:513] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.439612 14541 leader_election.cc:304] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81411e2807144c1aba3981c03d6c69a4; no voters:
I20250904 13:59:12.440263 14540 raft_consensus.cc:491] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:12.440657 14540 raft_consensus.cc:513] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.441165 14541 leader_election.cc:290] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:59:12.441255 14540 leader_election.cc:304] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81411e2807144c1aba3981c03d6c69a4; no voters:
I20250904 13:59:12.441500 14549 raft_consensus.cc:2802] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:12.441965 14540 leader_election.cc:290] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20250904 13:59:12.442080 14541 raft_consensus.cc:2802] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:12.443027 14549 raft_consensus.cc:695] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 [term 1 LEADER]: Becoming Leader. State: Replica: 81411e2807144c1aba3981c03d6c69a4, State: Running, Role: LEADER
I20250904 13:59:12.442531 14541 raft_consensus.cc:491] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:12.443632 14541 raft_consensus.cc:3058] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:12.443800 14549 consensus_queue.cc:237] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.450296 14383 catalog_manager.cc:5582] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81411e2807144c1aba3981c03d6c69a4 (127.10.98.129). New cstate: current_term: 1 leader_uuid: "81411e2807144c1aba3981c03d6c69a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } health_report { overall_health: HEALTHY } } }
I20250904 13:59:12.450565 14541 raft_consensus.cc:513] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.451256 14541 leader_election.cc:304] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81411e2807144c1aba3981c03d6c69a4; no voters:
I20250904 13:59:12.451779 14541 leader_election.cc:290] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:59:12.451923 14549 raft_consensus.cc:2802] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:12.452459 14549 raft_consensus.cc:695] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 [term 1 LEADER]: Becoming Leader. State: Replica: 81411e2807144c1aba3981c03d6c69a4, State: Running, Role: LEADER
I20250904 13:59:12.453138 14549 consensus_queue.cc:237] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.463547 14383 catalog_manager.cc:5582] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81411e2807144c1aba3981c03d6c69a4 (127.10.98.129). New cstate: current_term: 1 leader_uuid: "81411e2807144c1aba3981c03d6c69a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } health_report { overall_health: HEALTHY } } }
I20250904 13:59:12.588970 14549 raft_consensus.cc:491] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:12.589449 14549 raft_consensus.cc:513] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.590085 14549 leader_election.cc:304] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81411e2807144c1aba3981c03d6c69a4; no voters:
I20250904 13:59:12.590628 14549 leader_election.cc:290] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20250904 13:59:12.590807 14541 raft_consensus.cc:2802] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:12.591176 14541 raft_consensus.cc:491] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:12.591423 14541 raft_consensus.cc:3058] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:12.596711 14541 raft_consensus.cc:513] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.597285 14541 leader_election.cc:304] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81411e2807144c1aba3981c03d6c69a4; no voters:
I20250904 13:59:12.597808 14541 leader_election.cc:290] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:59:12.598057 14549 raft_consensus.cc:2802] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:12.598523 14549 raft_consensus.cc:695] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 [term 1 LEADER]: Becoming Leader. State: Replica: 81411e2807144c1aba3981c03d6c69a4, State: Running, Role: LEADER
I20250904 13:59:12.599107 14549 consensus_queue.cc:237] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.608464 14383 catalog_manager.cc:5582] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81411e2807144c1aba3981c03d6c69a4 (127.10.98.129). New cstate: current_term: 1 leader_uuid: "81411e2807144c1aba3981c03d6c69a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } health_report { overall_health: HEALTHY } } }
I20250904 13:59:12.625317 14549 raft_consensus.cc:491] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:12.625819 14549 raft_consensus.cc:513] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.626457 14549 leader_election.cc:304] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81411e2807144c1aba3981c03d6c69a4; no voters:
I20250904 13:59:12.626965 14549 leader_election.cc:290] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20250904 13:59:12.627084 14541 raft_consensus.cc:2802] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:12.627463 14541 raft_consensus.cc:491] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:12.627709 14541 raft_consensus.cc:3058] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:12.633237 14541 raft_consensus.cc:513] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.633852 14541 leader_election.cc:304] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81411e2807144c1aba3981c03d6c69a4; no voters:
I20250904 13:59:12.634375 14541 leader_election.cc:290] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:59:12.634501 14549 raft_consensus.cc:2802] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:12.634979 14549 raft_consensus.cc:695] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 [term 1 LEADER]: Becoming Leader. State: Replica: 81411e2807144c1aba3981c03d6c69a4, State: Running, Role: LEADER
I20250904 13:59:12.635811 14549 consensus_queue.cc:237] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.642335 14383 catalog_manager.cc:5582] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81411e2807144c1aba3981c03d6c69a4 (127.10.98.129). New cstate: current_term: 1 leader_uuid: "81411e2807144c1aba3981c03d6c69a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } health_report { overall_health: HEALTHY } } }
I20250904 13:59:12.688050 14549 raft_consensus.cc:491] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:12.688510 14549 raft_consensus.cc:513] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.689126 14549 leader_election.cc:304] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81411e2807144c1aba3981c03d6c69a4; no voters:
I20250904 13:59:12.689646 14549 leader_election.cc:290] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20250904 13:59:12.689749 14541 raft_consensus.cc:2802] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:12.690184 14541 raft_consensus.cc:491] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:12.690433 14541 raft_consensus.cc:3058] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:12.695721 14541 raft_consensus.cc:513] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.696311 14541 leader_election.cc:304] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81411e2807144c1aba3981c03d6c69a4; no voters:
I20250904 13:59:12.696811 14541 leader_election.cc:290] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:59:12.696951 14549 raft_consensus.cc:2802] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:12.697422 14549 raft_consensus.cc:695] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 [term 1 LEADER]: Becoming Leader. State: Replica: 81411e2807144c1aba3981c03d6c69a4, State: Running, Role: LEADER
I20250904 13:59:12.698083 14549 consensus_queue.cc:237] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.704181 14383 catalog_manager.cc:5582] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81411e2807144c1aba3981c03d6c69a4 (127.10.98.129). New cstate: current_term: 1 leader_uuid: "81411e2807144c1aba3981c03d6c69a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } health_report { overall_health: HEALTHY } } }
I20250904 13:59:12.708092 14549 raft_consensus.cc:491] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:12.708592 14549 raft_consensus.cc:513] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.709139 14549 leader_election.cc:304] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81411e2807144c1aba3981c03d6c69a4; no voters:
I20250904 13:59:12.709656 14549 leader_election.cc:290] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20250904 13:59:12.709765 14541 raft_consensus.cc:2802] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:12.710196 14541 raft_consensus.cc:491] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:12.710419 14549 raft_consensus.cc:491] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:12.710546 14541 raft_consensus.cc:3058] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:12.711011 14549 raft_consensus.cc:513] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.711727 14549 leader_election.cc:304] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81411e2807144c1aba3981c03d6c69a4; no voters:
I20250904 13:59:12.712203 14549 leader_election.cc:290] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20250904 13:59:12.712337 14550 raft_consensus.cc:2802] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:12.712749 14550 raft_consensus.cc:491] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:12.713042 14550 raft_consensus.cc:3058] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:12.716917 14541 raft_consensus.cc:513] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.717569 14541 leader_election.cc:304] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81411e2807144c1aba3981c03d6c69a4; no voters:
I20250904 13:59:12.718063 14541 leader_election.cc:290] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:59:12.718255 14549 raft_consensus.cc:2802] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:12.718784 14549 raft_consensus.cc:695] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 [term 1 LEADER]: Becoming Leader. State: Replica: 81411e2807144c1aba3981c03d6c69a4, State: Running, Role: LEADER
I20250904 13:59:12.719113 14550 raft_consensus.cc:513] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.719843 14550 leader_election.cc:304] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81411e2807144c1aba3981c03d6c69a4; no voters:
I20250904 13:59:12.719651 14549 consensus_queue.cc:237] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.720476 14550 leader_election.cc:290] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:59:12.720572 14541 raft_consensus.cc:2802] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:12.721064 14541 raft_consensus.cc:695] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 [term 1 LEADER]: Becoming Leader. State: Replica: 81411e2807144c1aba3981c03d6c69a4, State: Running, Role: LEADER
I20250904 13:59:12.721639 14541 consensus_queue.cc:237] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.726976 14383 catalog_manager.cc:5582] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81411e2807144c1aba3981c03d6c69a4 (127.10.98.129). New cstate: current_term: 1 leader_uuid: "81411e2807144c1aba3981c03d6c69a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } health_report { overall_health: HEALTHY } } }
I20250904 13:59:12.738859 14383 catalog_manager.cc:5582] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81411e2807144c1aba3981c03d6c69a4 (127.10.98.129). New cstate: current_term: 1 leader_uuid: "81411e2807144c1aba3981c03d6c69a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } health_report { overall_health: HEALTHY } } }
I20250904 13:59:12.881688 14541 raft_consensus.cc:491] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:12.882570 14541 raft_consensus.cc:513] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.883447 14541 leader_election.cc:304] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81411e2807144c1aba3981c03d6c69a4; no voters:
I20250904 13:59:12.884088 14541 leader_election.cc:290] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20250904 13:59:12.884335 14549 raft_consensus.cc:2802] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:12.884694 14549 raft_consensus.cc:491] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:12.884931 14549 raft_consensus.cc:3058] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:12.894758 14549 raft_consensus.cc:513] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.895409 14549 leader_election.cc:304] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81411e2807144c1aba3981c03d6c69a4; no voters:
I20250904 13:59:12.895920 14549 leader_election.cc:290] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:59:12.896078 14541 raft_consensus.cc:2802] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:12.896554 14541 raft_consensus.cc:695] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 [term 1 LEADER]: Becoming Leader. State: Replica: 81411e2807144c1aba3981c03d6c69a4, State: Running, Role: LEADER
I20250904 13:59:12.897758 14541 consensus_queue.cc:237] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } }
I20250904 13:59:12.906599 14383 catalog_manager.cc:5582] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 81411e2807144c1aba3981c03d6c69a4 (127.10.98.129). New cstate: current_term: 1 leader_uuid: "81411e2807144c1aba3981c03d6c69a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81411e2807144c1aba3981c03d6c69a4" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43537 } health_report { overall_health: HEALTHY } } }
I20250904 13:59:13.825989 10634 tablet_server.cc:178] TabletServer@127.10.98.129:0 shutting down...
I20250904 13:59:13.851965 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:59:13.852845 10634 tablet_replica.cc:331] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4: stopping tablet replica
I20250904 13:59:13.853701 10634 raft_consensus.cc:2241] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:13.854383 10634 raft_consensus.cc:2270] T 6c821980dec14c0a968c6f55ace0fc3f P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:13.858644 10634 tablet_replica.cc:331] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4: stopping tablet replica
I20250904 13:59:13.859375 10634 raft_consensus.cc:2241] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:13.859910 10634 raft_consensus.cc:2270] T e0bf641b345443a4a37326c8c5c346ff P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:13.862253 10634 tablet_replica.cc:331] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4: stopping tablet replica
I20250904 13:59:13.862906 10634 raft_consensus.cc:2241] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:13.863420 10634 raft_consensus.cc:2270] T c0e5555020ff4f06a76e21ba21285500 P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:13.865756 10634 tablet_replica.cc:331] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4: stopping tablet replica
I20250904 13:59:13.866410 10634 raft_consensus.cc:2241] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:13.866948 10634 raft_consensus.cc:2270] T ba40b00cc1614ca1aedbf6852db62c27 P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:13.869200 10634 tablet_replica.cc:331] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4: stopping tablet replica
I20250904 13:59:13.869786 10634 raft_consensus.cc:2241] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:13.870357 10634 raft_consensus.cc:2270] T 73d3f377465c4519a822c65abe9cdd36 P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:13.872803 10634 tablet_replica.cc:331] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4: stopping tablet replica
I20250904 13:59:13.873337 10634 raft_consensus.cc:2241] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:13.873880 10634 raft_consensus.cc:2270] T ccd9f92e96034550a059b0240c4abd44 P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:13.876149 10634 tablet_replica.cc:331] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4: stopping tablet replica
I20250904 13:59:13.876678 10634 raft_consensus.cc:2241] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:13.877193 10634 raft_consensus.cc:2270] T 2ffc529566c94e2e929beebb254c5ea1 P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:13.879397 10634 tablet_replica.cc:331] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4: stopping tablet replica
I20250904 13:59:13.879954 10634 raft_consensus.cc:2241] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:13.880566 10634 raft_consensus.cc:2270] T 5c1b17ca1fa345ca8c59f939422bb44f P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:13.882934 10634 tablet_replica.cc:331] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4: stopping tablet replica
I20250904 13:59:13.883473 10634 raft_consensus.cc:2241] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:13.884028 10634 raft_consensus.cc:2270] T 7a621adfa43047bfa1feb039ded81dac P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:13.886253 10634 tablet_replica.cc:331] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4: stopping tablet replica
I20250904 13:59:13.886734 10634 raft_consensus.cc:2241] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:13.887173 10634 raft_consensus.cc:2270] T 4b92eab78a464ac7be3d33e4da63d488 P 81411e2807144c1aba3981c03d6c69a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:13.919226 10634 tablet_server.cc:195] TabletServer@127.10.98.129:0 shutdown complete.
I20250904 13:59:13.947278 10634 master.cc:561] Master@127.10.98.190:38271 shutting down...
I20250904 13:59:13.966985 10634 raft_consensus.cc:2241] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:13.967592 10634 raft_consensus.cc:2270] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:13.967947 10634 tablet_replica.cc:331] T 00000000000000000000000000000000 P fb223b9a51754aac9c57648a6d1efa20: stopping tablet replica
I20250904 13:59:13.988008 10634 master.cc:583] Master@127.10.98.190:38271 shutdown complete.
[ OK ] NumReplicas/TServerQuiescingParamITest.TestNoElectionsForNewReplicas/0 (3925 ms)
[ RUN ] NumReplicas/TServerQuiescingParamITest.TestNoElectionsForNewReplicas/1
I20250904 13:59:14.028999 10634 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.98.190:34049
I20250904 13:59:14.030345 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:14.036175 14560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:14.037436 14561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:14.037881 14563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:14.038715 10634 server_base.cc:1047] running on GCE node
I20250904 13:59:14.039568 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:14.039762 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:14.039937 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994354039915 us; error 0 us; skew 500 ppm
I20250904 13:59:14.040432 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:14.043068 10634 webserver.cc:480] Webserver started at http://127.10.98.190:38779/ using document root <none> and password file <none>
I20250904 13:59:14.043612 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:14.043789 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:14.044070 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:14.045295 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_1.1756994241738102-10634-0/minicluster-data/master-0-root/instance:
uuid: "3bd50f780abc4ef497d8a29d5d5feac2"
format_stamp: "Formatted at 2025-09-04 13:59:14 on dist-test-slave-jx76"
I20250904 13:59:14.050093 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20250904 13:59:14.053759 14568 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:14.054574 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20250904 13:59:14.054852 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_1.1756994241738102-10634-0/minicluster-data/master-0-root
uuid: "3bd50f780abc4ef497d8a29d5d5feac2"
format_stamp: "Formatted at 2025-09-04 13:59:14 on dist-test-slave-jx76"
I20250904 13:59:14.055171 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_1.1756994241738102-10634-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_1.1756994241738102-10634-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_1.1756994241738102-10634-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:14.070686 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:14.071983 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:14.113600 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.190:34049
I20250904 13:59:14.113724 14619 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.190:34049 every 8 connection(s)
I20250904 13:59:14.117765 14620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.130962 14620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2: Bootstrap starting.
I20250904 13:59:14.136109 14620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:14.140642 14620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2: No bootstrap required, opened a new log
I20250904 13:59:14.142920 14620 raft_consensus.cc:357] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd50f780abc4ef497d8a29d5d5feac2" member_type: VOTER }
I20250904 13:59:14.143325 14620 raft_consensus.cc:383] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:14.143527 14620 raft_consensus.cc:738] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bd50f780abc4ef497d8a29d5d5feac2, State: Initialized, Role: FOLLOWER
I20250904 13:59:14.144127 14620 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd50f780abc4ef497d8a29d5d5feac2" member_type: VOTER }
I20250904 13:59:14.144559 14620 raft_consensus.cc:397] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:59:14.144749 14620 raft_consensus.cc:491] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250904 13:59:14.144966 14620 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:14.150460 14620 raft_consensus.cc:513] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd50f780abc4ef497d8a29d5d5feac2" member_type: VOTER }
I20250904 13:59:14.151018 14620 leader_election.cc:304] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3bd50f780abc4ef497d8a29d5d5feac2; no voters:
I20250904 13:59:14.152201 14620 leader_election.cc:290] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:59:14.152693 14623 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:14.154145 14623 raft_consensus.cc:695] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2 [term 1 LEADER]: Becoming Leader. State: Replica: 3bd50f780abc4ef497d8a29d5d5feac2, State: Running, Role: LEADER
I20250904 13:59:14.154964 14623 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd50f780abc4ef497d8a29d5d5feac2" member_type: VOTER }
I20250904 13:59:14.155522 14620 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 13:59:14.158207 14625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3bd50f780abc4ef497d8a29d5d5feac2. Latest consensus state: current_term: 1 leader_uuid: "3bd50f780abc4ef497d8a29d5d5feac2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd50f780abc4ef497d8a29d5d5feac2" member_type: VOTER } }
I20250904 13:59:14.158205 14624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3bd50f780abc4ef497d8a29d5d5feac2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3bd50f780abc4ef497d8a29d5d5feac2" member_type: VOTER } }
I20250904 13:59:14.158985 14625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2 [sys.catalog]: This master's current role is: LEADER
I20250904 13:59:14.159191 14624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2 [sys.catalog]: This master's current role is: LEADER
I20250904 13:59:14.163421 14628 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 13:59:14.171609 14628 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 13:59:14.172672 10634 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 13:59:14.181196 14628 catalog_manager.cc:1349] Generated new cluster ID: ca6e167c8308471389a15432167c4d82
I20250904 13:59:14.181483 14628 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 13:59:14.197180 14628 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 13:59:14.198788 14628 catalog_manager.cc:1506] Loading token signing keys...
I20250904 13:59:14.216013 14628 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2: Generated new TSK 0
I20250904 13:59:14.216746 14628 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 13:59:14.239576 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:14.245792 14641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:14.247239 14642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:14.248683 14644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:14.249441 10634 server_base.cc:1047] running on GCE node
I20250904 13:59:14.250367 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:14.250558 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:14.250684 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994354250671 us; error 0 us; skew 500 ppm
I20250904 13:59:14.251118 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:14.253502 10634 webserver.cc:480] Webserver started at http://127.10.98.129:38703/ using document root <none> and password file <none>
I20250904 13:59:14.254014 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:14.254246 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:14.254508 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:14.255861 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_1.1756994241738102-10634-0/minicluster-data/ts-0-root/instance:
uuid: "1f0cdd7953ea4648a402cfaaa28ce748"
format_stamp: "Formatted at 2025-09-04 13:59:14 on dist-test-slave-jx76"
I20250904 13:59:14.261152 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20250904 13:59:14.264915 14649 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:14.265750 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250904 13:59:14.266049 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_1.1756994241738102-10634-0/minicluster-data/ts-0-root
uuid: "1f0cdd7953ea4648a402cfaaa28ce748"
format_stamp: "Formatted at 2025-09-04 13:59:14 on dist-test-slave-jx76"
I20250904 13:59:14.266431 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_1.1756994241738102-10634-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_1.1756994241738102-10634-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_1.1756994241738102-10634-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:14.285195 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:14.286478 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:14.288059 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:59:14.290522 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:59:14.290743 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:14.290987 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:59:14.291160 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:14.335176 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.129:43873
I20250904 13:59:14.335290 14711 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.129:43873 every 8 connection(s)
I20250904 13:59:14.340286 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:14.348824 14716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:14.349809 14717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:14.352640 14719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:14.352979 10634 server_base.cc:1047] running on GCE node
I20250904 13:59:14.353955 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:14.354198 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:14.354336 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994354354324 us; error 0 us; skew 500 ppm
I20250904 13:59:14.354804 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:14.357312 10634 webserver.cc:480] Webserver started at http://127.10.98.130:34243/ using document root <none> and password file <none>
I20250904 13:59:14.357817 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:14.357990 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:14.358274 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:14.359550 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_1.1756994241738102-10634-0/minicluster-data/ts-1-root/instance:
uuid: "6b21299357f942dab7c574db37593c3b"
format_stamp: "Formatted at 2025-09-04 13:59:14 on dist-test-slave-jx76"
I20250904 13:59:14.363881 14712 heartbeater.cc:344] Connected to a master server at 127.10.98.190:34049
I20250904 13:59:14.364342 14712 heartbeater.cc:461] Registering TS with master...
I20250904 13:59:14.365468 14712 heartbeater.cc:507] Master 127.10.98.190:34049 requested a full tablet report, sending...
I20250904 13:59:14.365520 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20250904 13:59:14.367736 14585 ts_manager.cc:194] Registered new tserver with Master: 1f0cdd7953ea4648a402cfaaa28ce748 (127.10.98.129:43873)
I20250904 13:59:14.369867 14585 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43254
I20250904 13:59:14.369980 14724 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:14.370999 10634 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20250904 13:59:14.371291 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_1.1756994241738102-10634-0/minicluster-data/ts-1-root
uuid: "6b21299357f942dab7c574db37593c3b"
format_stamp: "Formatted at 2025-09-04 13:59:14 on dist-test-slave-jx76"
I20250904 13:59:14.371690 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_1.1756994241738102-10634-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_1.1756994241738102-10634-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_1.1756994241738102-10634-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:14.395182 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:14.396445 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:14.397964 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:59:14.400502 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:59:14.400698 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:14.400981 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:59:14.401129 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:14.444379 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.130:42551
I20250904 13:59:14.444494 14786 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.130:42551 every 8 connection(s)
I20250904 13:59:14.449496 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:14.456962 14790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:14.458437 14791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:14.463256 14787 heartbeater.cc:344] Connected to a master server at 127.10.98.190:34049
I20250904 13:59:14.463826 14787 heartbeater.cc:461] Registering TS with master...
I20250904 13:59:14.464723 10634 server_base.cc:1047] running on GCE node
I20250904 13:59:14.465023 14787 heartbeater.cc:507] Master 127.10.98.190:34049 requested a full tablet report, sending...
W20250904 13:59:14.465389 14794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:14.466413 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:14.466646 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:14.466825 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994354466803 us; error 0 us; skew 500 ppm
I20250904 13:59:14.467499 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:14.467625 14585 ts_manager.cc:194] Registered new tserver with Master: 6b21299357f942dab7c574db37593c3b (127.10.98.130:42551)
I20250904 13:59:14.469679 14585 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43260
I20250904 13:59:14.470482 10634 webserver.cc:480] Webserver started at http://127.10.98.131:34043/ using document root <none> and password file <none>
I20250904 13:59:14.471161 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:14.471413 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:14.471761 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:14.473229 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_1.1756994241738102-10634-0/minicluster-data/ts-2-root/instance:
uuid: "eae0557778094d6c9e874fb9f99ead31"
format_stamp: "Formatted at 2025-09-04 13:59:14 on dist-test-slave-jx76"
I20250904 13:59:14.478535 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20250904 13:59:14.482065 14798 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:14.482934 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250904 13:59:14.483297 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_1.1756994241738102-10634-0/minicluster-data/ts-2-root
uuid: "eae0557778094d6c9e874fb9f99ead31"
format_stamp: "Formatted at 2025-09-04 13:59:14 on dist-test-slave-jx76"
I20250904 13:59:14.483603 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_1.1756994241738102-10634-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_1.1756994241738102-10634-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestNoElectionsForNewReplicas_1.1756994241738102-10634-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:14.500476 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:14.501761 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:14.503374 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:59:14.505832 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:59:14.506026 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:14.506327 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:59:14.506475 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:14.548368 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.131:44951
I20250904 13:59:14.548480 14860 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.131:44951 every 8 connection(s)
I20250904 13:59:14.563699 14861 heartbeater.cc:344] Connected to a master server at 127.10.98.190:34049
I20250904 13:59:14.564118 14861 heartbeater.cc:461] Registering TS with master...
I20250904 13:59:14.564901 14861 heartbeater.cc:507] Master 127.10.98.190:34049 requested a full tablet report, sending...
I20250904 13:59:14.566967 14585 ts_manager.cc:194] Registered new tserver with Master: eae0557778094d6c9e874fb9f99ead31 (127.10.98.131:44951)
I20250904 13:59:14.567575 10634 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015781148s
I20250904 13:59:14.568560 14585 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43276
I20250904 13:59:14.603014 10634 test_util.cc:276] Using random seed: 543220711
I20250904 13:59:14.629415 14585 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43318:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 13:59:14.632572 14585 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 13:59:14.709896 14752 tablet_service.cc:1468] Processing CreateTablet for tablet 4903d9835fcc4652b77ae055c23b8a4d (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION VALUES < 214748364
I20250904 13:59:14.711537 14752 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4903d9835fcc4652b77ae055c23b8a4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.716220 14677 tablet_service.cc:1468] Processing CreateTablet for tablet 4903d9835fcc4652b77ae055c23b8a4d (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION VALUES < 214748364
I20250904 13:59:14.717788 14677 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4903d9835fcc4652b77ae055c23b8a4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.719476 14676 tablet_service.cc:1468] Processing CreateTablet for tablet 77ea8776d09e48978e66a1c36c15cb56 (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20250904 13:59:14.722556 14674 tablet_service.cc:1468] Processing CreateTablet for tablet 68be2a2fe36345b4aaafe10bce4ab6cb (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20250904 13:59:14.724062 14674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68be2a2fe36345b4aaafe10bce4ab6cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.724262 14750 tablet_service.cc:1468] Processing CreateTablet for tablet a93701c10d644072a8889318fd08c99a (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20250904 13:59:14.725711 14750 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a93701c10d644072a8889318fd08c99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.725878 14747 tablet_service.cc:1468] Processing CreateTablet for tablet 2746477d17a44175a64aed903c4cc706 (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20250904 13:59:14.727577 14747 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2746477d17a44175a64aed903c4cc706. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.727272 14675 tablet_service.cc:1468] Processing CreateTablet for tablet a93701c10d644072a8889318fd08c99a (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20250904 13:59:14.728700 14675 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a93701c10d644072a8889318fd08c99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.729283 14823 tablet_service.cc:1468] Processing CreateTablet for tablet 68be2a2fe36345b4aaafe10bce4ab6cb (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20250904 13:59:14.730746 14823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68be2a2fe36345b4aaafe10bce4ab6cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.731068 14751 tablet_service.cc:1468] Processing CreateTablet for tablet 77ea8776d09e48978e66a1c36c15cb56 (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20250904 13:59:14.732502 14751 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77ea8776d09e48978e66a1c36c15cb56. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.732453 14745 tablet_service.cc:1468] Processing CreateTablet for tablet 98d004e260ad46168f1321f718eef6a1 (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20250904 13:59:14.735357 14748 tablet_service.cc:1468] Processing CreateTablet for tablet c69c4a3a981847d8a5febe6a619cc8a1 (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20250904 13:59:14.736711 14824 tablet_service.cc:1468] Processing CreateTablet for tablet a93701c10d644072a8889318fd08c99a (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20250904 13:59:14.739392 14746 tablet_service.cc:1468] Processing CreateTablet for tablet f1fb57f4c29f42f3901517c38d4fd2bd (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20250904 13:59:14.740777 14746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1fb57f4c29f42f3901517c38d4fd2bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.743391 14743 tablet_service.cc:1468] Processing CreateTablet for tablet 4eb02492ca4c4f0f990cb55611347087 (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20250904 13:59:14.745601 14818 tablet_service.cc:1468] Processing CreateTablet for tablet 38244a7426a443efbd07ab8975ae0b75 (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20250904 13:59:14.745882 14822 tablet_service.cc:1468] Processing CreateTablet for tablet c69c4a3a981847d8a5febe6a619cc8a1 (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20250904 13:59:14.748309 14821 tablet_service.cc:1468] Processing CreateTablet for tablet 2746477d17a44175a64aed903c4cc706 (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20250904 13:59:14.751474 14668 tablet_service.cc:1468] Processing CreateTablet for tablet 4eb02492ca4c4f0f990cb55611347087 (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20250904 13:59:14.751338 14820 tablet_service.cc:1468] Processing CreateTablet for tablet f1fb57f4c29f42f3901517c38d4fd2bd (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20250904 13:59:14.752964 14668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eb02492ca4c4f0f990cb55611347087. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.755100 14676 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77ea8776d09e48978e66a1c36c15cb56. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.755481 14669 tablet_service.cc:1468] Processing CreateTablet for tablet 38244a7426a443efbd07ab8975ae0b75 (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20250904 13:59:14.722455 14749 tablet_service.cc:1468] Processing CreateTablet for tablet 68be2a2fe36345b4aaafe10bce4ab6cb (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20250904 13:59:14.743842 14748 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c69c4a3a981847d8a5febe6a619cc8a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.755739 14819 tablet_service.cc:1468] Processing CreateTablet for tablet 98d004e260ad46168f1321f718eef6a1 (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20250904 13:59:14.756783 14824 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a93701c10d644072a8889318fd08c99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.759033 14817 tablet_service.cc:1468] Processing CreateTablet for tablet 4eb02492ca4c4f0f990cb55611347087 (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20250904 13:59:14.768076 14671 tablet_service.cc:1468] Processing CreateTablet for tablet f1fb57f4c29f42f3901517c38d4fd2bd (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20250904 13:59:14.768546 14817 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eb02492ca4c4f0f990cb55611347087. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.769554 14822 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c69c4a3a981847d8a5febe6a619cc8a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.771133 14670 tablet_service.cc:1468] Processing CreateTablet for tablet 98d004e260ad46168f1321f718eef6a1 (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20250904 13:59:14.773777 14744 tablet_service.cc:1468] Processing CreateTablet for tablet 38244a7426a443efbd07ab8975ae0b75 (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20250904 13:59:14.776153 14826 tablet_service.cc:1468] Processing CreateTablet for tablet 4903d9835fcc4652b77ae055c23b8a4d (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION VALUES < 214748364
I20250904 13:59:14.777612 14826 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4903d9835fcc4652b77ae055c23b8a4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.778645 14820 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1fb57f4c29f42f3901517c38d4fd2bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.782799 14743 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4eb02492ca4c4f0f990cb55611347087. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.735414 14825 tablet_service.cc:1468] Processing CreateTablet for tablet 77ea8776d09e48978e66a1c36c15cb56 (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20250904 13:59:14.785794 14825 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77ea8776d09e48978e66a1c36c15cb56. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.785939 14745 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98d004e260ad46168f1321f718eef6a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.791033 14819 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98d004e260ad46168f1321f718eef6a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.795047 14749 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68be2a2fe36345b4aaafe10bce4ab6cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.763826 14672 tablet_service.cc:1468] Processing CreateTablet for tablet 2746477d17a44175a64aed903c4cc706 (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20250904 13:59:14.798996 14669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38244a7426a443efbd07ab8975ae0b75. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.761529 14673 tablet_service.cc:1468] Processing CreateTablet for tablet c69c4a3a981847d8a5febe6a619cc8a1 (DEFAULT_TABLE table=test-workload [id=d8122de0a1f245aaafd61eff74655bfc]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20250904 13:59:14.803411 14744 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38244a7426a443efbd07ab8975ae0b75. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.803411 14670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98d004e260ad46168f1321f718eef6a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.807610 14673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c69c4a3a981847d8a5febe6a619cc8a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.810509 14672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2746477d17a44175a64aed903c4cc706. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.814538 14671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f1fb57f4c29f42f3901517c38d4fd2bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.821784 14821 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2746477d17a44175a64aed903c4cc706. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.824898 14818 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38244a7426a443efbd07ab8975ae0b75. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:14.883000 14883 tablet_bootstrap.cc:492] T 4eb02492ca4c4f0f990cb55611347087 P eae0557778094d6c9e874fb9f99ead31: Bootstrap starting.
I20250904 13:59:14.928783 14882 tablet_bootstrap.cc:492] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748: Bootstrap starting.
I20250904 13:59:14.941499 14884 tablet_bootstrap.cc:492] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b: Bootstrap starting.
I20250904 13:59:14.943647 14882 tablet_bootstrap.cc:654] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:14.943647 14883 tablet_bootstrap.cc:654] T 4eb02492ca4c4f0f990cb55611347087 P eae0557778094d6c9e874fb9f99ead31: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:14.956743 14884 tablet_bootstrap.cc:654] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:14.990949 14883 tablet_bootstrap.cc:492] T 4eb02492ca4c4f0f990cb55611347087 P eae0557778094d6c9e874fb9f99ead31: No bootstrap required, opened a new log
I20250904 13:59:14.991494 14883 ts_tablet_manager.cc:1397] T 4eb02492ca4c4f0f990cb55611347087 P eae0557778094d6c9e874fb9f99ead31: Time spent bootstrapping tablet: real 0.109s user 0.023s sys 0.008s
I20250904 13:59:14.994314 14883 raft_consensus.cc:357] T 4eb02492ca4c4f0f990cb55611347087 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:14.995061 14883 raft_consensus.cc:383] T 4eb02492ca4c4f0f990cb55611347087 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:14.995404 14883 raft_consensus.cc:738] T 4eb02492ca4c4f0f990cb55611347087 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eae0557778094d6c9e874fb9f99ead31, State: Initialized, Role: FOLLOWER
I20250904 13:59:14.996280 14883 consensus_queue.cc:260] T 4eb02492ca4c4f0f990cb55611347087 P eae0557778094d6c9e874fb9f99ead31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.002059 14861 heartbeater.cc:499] Master 127.10.98.190:34049 was elected leader, sending a full tablet report...
I20250904 13:59:15.017490 14883 ts_tablet_manager.cc:1428] T 4eb02492ca4c4f0f990cb55611347087 P eae0557778094d6c9e874fb9f99ead31: Time spent starting tablet: real 0.026s user 0.009s sys 0.000s
I20250904 13:59:15.020128 14883 tablet_bootstrap.cc:492] T a93701c10d644072a8889318fd08c99a P eae0557778094d6c9e874fb9f99ead31: Bootstrap starting.
I20250904 13:59:15.022388 14884 tablet_bootstrap.cc:492] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b: No bootstrap required, opened a new log
I20250904 13:59:15.024050 14882 tablet_bootstrap.cc:492] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748: No bootstrap required, opened a new log
I20250904 13:59:15.026587 14884 ts_tablet_manager.cc:1397] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b: Time spent bootstrapping tablet: real 0.085s user 0.012s sys 0.016s
I20250904 13:59:15.026683 14882 ts_tablet_manager.cc:1397] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748: Time spent bootstrapping tablet: real 0.098s user 0.013s sys 0.014s
I20250904 13:59:15.026041 14883 tablet_bootstrap.cc:654] T a93701c10d644072a8889318fd08c99a P eae0557778094d6c9e874fb9f99ead31: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.029809 14882 raft_consensus.cc:357] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.030066 14884 raft_consensus.cc:357] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.030844 14882 raft_consensus.cc:383] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.030937 14884 raft_consensus.cc:383] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.031322 14882 raft_consensus.cc:738] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0cdd7953ea4648a402cfaaa28ce748, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.031374 14884 raft_consensus.cc:738] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b21299357f942dab7c574db37593c3b, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.032589 14884 consensus_queue.cc:260] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.032658 14882 consensus_queue.cc:260] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.055933 14787 heartbeater.cc:499] Master 127.10.98.190:34049 was elected leader, sending a full tablet report...
I20250904 13:59:15.057235 14883 tablet_bootstrap.cc:492] T a93701c10d644072a8889318fd08c99a P eae0557778094d6c9e874fb9f99ead31: No bootstrap required, opened a new log
I20250904 13:59:15.057266 14712 heartbeater.cc:499] Master 127.10.98.190:34049 was elected leader, sending a full tablet report...
I20250904 13:59:15.057832 14882 ts_tablet_manager.cc:1428] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748: Time spent starting tablet: real 0.031s user 0.013s sys 0.017s
I20250904 13:59:15.057857 14883 ts_tablet_manager.cc:1397] T a93701c10d644072a8889318fd08c99a P eae0557778094d6c9e874fb9f99ead31: Time spent bootstrapping tablet: real 0.038s user 0.026s sys 0.008s
I20250904 13:59:15.058949 14882 tablet_bootstrap.cc:492] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748: Bootstrap starting.
I20250904 13:59:15.061040 14883 raft_consensus.cc:357] T a93701c10d644072a8889318fd08c99a P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.061873 14883 raft_consensus.cc:383] T a93701c10d644072a8889318fd08c99a P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.062245 14883 raft_consensus.cc:738] T a93701c10d644072a8889318fd08c99a P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eae0557778094d6c9e874fb9f99ead31, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.063673 14883 consensus_queue.cc:260] T a93701c10d644072a8889318fd08c99a P eae0557778094d6c9e874fb9f99ead31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.066314 14883 ts_tablet_manager.cc:1428] T a93701c10d644072a8889318fd08c99a P eae0557778094d6c9e874fb9f99ead31: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20250904 13:59:15.067507 14883 tablet_bootstrap.cc:492] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31: Bootstrap starting.
I20250904 13:59:15.067081 14884 ts_tablet_manager.cc:1428] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b: Time spent starting tablet: real 0.040s user 0.009s sys 0.019s
I20250904 13:59:15.070328 14884 tablet_bootstrap.cc:492] T a93701c10d644072a8889318fd08c99a P 6b21299357f942dab7c574db37593c3b: Bootstrap starting.
I20250904 13:59:15.070823 14882 tablet_bootstrap.cc:654] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.074702 14883 tablet_bootstrap.cc:654] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.079591 14884 tablet_bootstrap.cc:654] T a93701c10d644072a8889318fd08c99a P 6b21299357f942dab7c574db37593c3b: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.083348 14882 tablet_bootstrap.cc:492] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748: No bootstrap required, opened a new log
I20250904 13:59:15.084250 14882 ts_tablet_manager.cc:1397] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748: Time spent bootstrapping tablet: real 0.026s user 0.012s sys 0.007s
I20250904 13:59:15.090319 14882 raft_consensus.cc:357] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.091756 14882 raft_consensus.cc:383] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.092257 14882 raft_consensus.cc:738] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0cdd7953ea4648a402cfaaa28ce748, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.093597 14882 consensus_queue.cc:260] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.098315 14882 ts_tablet_manager.cc:1428] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
I20250904 13:59:15.100370 14882 tablet_bootstrap.cc:492] T 2746477d17a44175a64aed903c4cc706 P 1f0cdd7953ea4648a402cfaaa28ce748: Bootstrap starting.
I20250904 13:59:15.101922 14884 tablet_bootstrap.cc:492] T a93701c10d644072a8889318fd08c99a P 6b21299357f942dab7c574db37593c3b: No bootstrap required, opened a new log
I20250904 13:59:15.102628 14884 ts_tablet_manager.cc:1397] T a93701c10d644072a8889318fd08c99a P 6b21299357f942dab7c574db37593c3b: Time spent bootstrapping tablet: real 0.033s user 0.015s sys 0.015s
I20250904 13:59:15.105827 14884 raft_consensus.cc:357] T a93701c10d644072a8889318fd08c99a P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.106770 14884 raft_consensus.cc:383] T a93701c10d644072a8889318fd08c99a P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.107142 14884 raft_consensus.cc:738] T a93701c10d644072a8889318fd08c99a P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b21299357f942dab7c574db37593c3b, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.107378 14882 tablet_bootstrap.cc:654] T 2746477d17a44175a64aed903c4cc706 P 1f0cdd7953ea4648a402cfaaa28ce748: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.108223 14884 consensus_queue.cc:260] T a93701c10d644072a8889318fd08c99a P 6b21299357f942dab7c574db37593c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.110913 14884 ts_tablet_manager.cc:1428] T a93701c10d644072a8889318fd08c99a P 6b21299357f942dab7c574db37593c3b: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20250904 13:59:15.112458 14884 tablet_bootstrap.cc:492] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b: Bootstrap starting.
I20250904 13:59:15.120188 14882 tablet_bootstrap.cc:492] T 2746477d17a44175a64aed903c4cc706 P 1f0cdd7953ea4648a402cfaaa28ce748: No bootstrap required, opened a new log
I20250904 13:59:15.120622 14884 tablet_bootstrap.cc:654] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.120945 14882 ts_tablet_manager.cc:1397] T 2746477d17a44175a64aed903c4cc706 P 1f0cdd7953ea4648a402cfaaa28ce748: Time spent bootstrapping tablet: real 0.021s user 0.018s sys 0.001s
I20250904 13:59:15.125283 14882 raft_consensus.cc:357] T 2746477d17a44175a64aed903c4cc706 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.126634 14882 raft_consensus.cc:383] T 2746477d17a44175a64aed903c4cc706 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.127187 14882 raft_consensus.cc:738] T 2746477d17a44175a64aed903c4cc706 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0cdd7953ea4648a402cfaaa28ce748, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.128278 14882 consensus_queue.cc:260] T 2746477d17a44175a64aed903c4cc706 P 1f0cdd7953ea4648a402cfaaa28ce748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.131565 14882 ts_tablet_manager.cc:1428] T 2746477d17a44175a64aed903c4cc706 P 1f0cdd7953ea4648a402cfaaa28ce748: Time spent starting tablet: real 0.010s user 0.007s sys 0.001s
I20250904 13:59:15.132778 14882 tablet_bootstrap.cc:492] T 68be2a2fe36345b4aaafe10bce4ab6cb P 1f0cdd7953ea4648a402cfaaa28ce748: Bootstrap starting.
I20250904 13:59:15.133644 14884 tablet_bootstrap.cc:492] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b: No bootstrap required, opened a new log
I20250904 13:59:15.134294 14884 ts_tablet_manager.cc:1397] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b: Time spent bootstrapping tablet: real 0.022s user 0.015s sys 0.004s
I20250904 13:59:15.137444 14884 raft_consensus.cc:357] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.138578 14884 raft_consensus.cc:383] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.138944 14884 raft_consensus.cc:738] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b21299357f942dab7c574db37593c3b, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.139657 14882 tablet_bootstrap.cc:654] T 68be2a2fe36345b4aaafe10bce4ab6cb P 1f0cdd7953ea4648a402cfaaa28ce748: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.139856 14884 consensus_queue.cc:260] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.143095 14884 ts_tablet_manager.cc:1428] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20250904 13:59:15.144611 14884 tablet_bootstrap.cc:492] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b: Bootstrap starting.
I20250904 13:59:15.151901 14882 tablet_bootstrap.cc:492] T 68be2a2fe36345b4aaafe10bce4ab6cb P 1f0cdd7953ea4648a402cfaaa28ce748: No bootstrap required, opened a new log
I20250904 13:59:15.152628 14884 tablet_bootstrap.cc:654] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.152666 14882 ts_tablet_manager.cc:1397] T 68be2a2fe36345b4aaafe10bce4ab6cb P 1f0cdd7953ea4648a402cfaaa28ce748: Time spent bootstrapping tablet: real 0.020s user 0.011s sys 0.008s
I20250904 13:59:15.159312 14882 raft_consensus.cc:357] T 68be2a2fe36345b4aaafe10bce4ab6cb P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.160585 14882 raft_consensus.cc:383] T 68be2a2fe36345b4aaafe10bce4ab6cb P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.161058 14882 raft_consensus.cc:738] T 68be2a2fe36345b4aaafe10bce4ab6cb P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0cdd7953ea4648a402cfaaa28ce748, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.162199 14883 tablet_bootstrap.cc:492] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31: No bootstrap required, opened a new log
I20250904 13:59:15.163003 14883 ts_tablet_manager.cc:1397] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31: Time spent bootstrapping tablet: real 0.096s user 0.047s sys 0.047s
I20250904 13:59:15.162245 14882 consensus_queue.cc:260] T 68be2a2fe36345b4aaafe10bce4ab6cb P 1f0cdd7953ea4648a402cfaaa28ce748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.166324 14882 ts_tablet_manager.cc:1428] T 68be2a2fe36345b4aaafe10bce4ab6cb P 1f0cdd7953ea4648a402cfaaa28ce748: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20250904 13:59:15.167459 14882 tablet_bootstrap.cc:492] T c69c4a3a981847d8a5febe6a619cc8a1 P 1f0cdd7953ea4648a402cfaaa28ce748: Bootstrap starting.
I20250904 13:59:15.167114 14883 raft_consensus.cc:357] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.168092 14883 raft_consensus.cc:383] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.168473 14883 raft_consensus.cc:738] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eae0557778094d6c9e874fb9f99ead31, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.168900 14884 tablet_bootstrap.cc:492] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b: No bootstrap required, opened a new log
I20250904 13:59:15.169459 14884 ts_tablet_manager.cc:1397] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.017s
I20250904 13:59:15.169412 14883 consensus_queue.cc:260] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.173102 14884 raft_consensus.cc:357] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.173986 14884 raft_consensus.cc:383] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.174373 14883 ts_tablet_manager.cc:1428] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31: Time spent starting tablet: real 0.011s user 0.007s sys 0.000s
I20250904 13:59:15.174389 14884 raft_consensus.cc:738] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b21299357f942dab7c574db37593c3b, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.175282 14882 tablet_bootstrap.cc:654] T c69c4a3a981847d8a5febe6a619cc8a1 P 1f0cdd7953ea4648a402cfaaa28ce748: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.175663 14883 tablet_bootstrap.cc:492] T 38244a7426a443efbd07ab8975ae0b75 P eae0557778094d6c9e874fb9f99ead31: Bootstrap starting.
I20250904 13:59:15.175618 14884 consensus_queue.cc:260] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.179088 14884 ts_tablet_manager.cc:1428] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20250904 13:59:15.180640 14884 tablet_bootstrap.cc:492] T 98d004e260ad46168f1321f718eef6a1 P 6b21299357f942dab7c574db37593c3b: Bootstrap starting.
I20250904 13:59:15.183085 14883 tablet_bootstrap.cc:654] T 38244a7426a443efbd07ab8975ae0b75 P eae0557778094d6c9e874fb9f99ead31: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.188153 14882 tablet_bootstrap.cc:492] T c69c4a3a981847d8a5febe6a619cc8a1 P 1f0cdd7953ea4648a402cfaaa28ce748: No bootstrap required, opened a new log
I20250904 13:59:15.188627 14884 tablet_bootstrap.cc:654] T 98d004e260ad46168f1321f718eef6a1 P 6b21299357f942dab7c574db37593c3b: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.188848 14882 ts_tablet_manager.cc:1397] T c69c4a3a981847d8a5febe6a619cc8a1 P 1f0cdd7953ea4648a402cfaaa28ce748: Time spent bootstrapping tablet: real 0.022s user 0.011s sys 0.008s
I20250904 13:59:15.192282 14882 raft_consensus.cc:357] T c69c4a3a981847d8a5febe6a619cc8a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.193748 14882 raft_consensus.cc:383] T c69c4a3a981847d8a5febe6a619cc8a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.194281 14882 raft_consensus.cc:738] T c69c4a3a981847d8a5febe6a619cc8a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0cdd7953ea4648a402cfaaa28ce748, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.195537 14882 consensus_queue.cc:260] T c69c4a3a981847d8a5febe6a619cc8a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.199174 14882 ts_tablet_manager.cc:1428] T c69c4a3a981847d8a5febe6a619cc8a1 P 1f0cdd7953ea4648a402cfaaa28ce748: Time spent starting tablet: real 0.010s user 0.007s sys 0.001s
I20250904 13:59:15.201133 14884 tablet_bootstrap.cc:492] T 98d004e260ad46168f1321f718eef6a1 P 6b21299357f942dab7c574db37593c3b: No bootstrap required, opened a new log
I20250904 13:59:15.201500 14882 tablet_bootstrap.cc:492] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748: Bootstrap starting.
I20250904 13:59:15.201740 14884 ts_tablet_manager.cc:1397] T 98d004e260ad46168f1321f718eef6a1 P 6b21299357f942dab7c574db37593c3b: Time spent bootstrapping tablet: real 0.021s user 0.015s sys 0.005s
I20250904 13:59:15.202201 14883 tablet_bootstrap.cc:492] T 38244a7426a443efbd07ab8975ae0b75 P eae0557778094d6c9e874fb9f99ead31: No bootstrap required, opened a new log
I20250904 13:59:15.202759 14883 ts_tablet_manager.cc:1397] T 38244a7426a443efbd07ab8975ae0b75 P eae0557778094d6c9e874fb9f99ead31: Time spent bootstrapping tablet: real 0.027s user 0.016s sys 0.003s
I20250904 13:59:15.205749 14883 raft_consensus.cc:357] T 38244a7426a443efbd07ab8975ae0b75 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.205174 14884 raft_consensus.cc:357] T 98d004e260ad46168f1321f718eef6a1 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.206673 14883 raft_consensus.cc:383] T 38244a7426a443efbd07ab8975ae0b75 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.206888 14884 raft_consensus.cc:383] T 98d004e260ad46168f1321f718eef6a1 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.207124 14883 raft_consensus.cc:738] T 38244a7426a443efbd07ab8975ae0b75 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eae0557778094d6c9e874fb9f99ead31, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.207341 14884 raft_consensus.cc:738] T 98d004e260ad46168f1321f718eef6a1 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b21299357f942dab7c574db37593c3b, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.208112 14883 consensus_queue.cc:260] T 38244a7426a443efbd07ab8975ae0b75 P eae0557778094d6c9e874fb9f99ead31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.208911 14882 tablet_bootstrap.cc:654] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.208730 14884 consensus_queue.cc:260] T 98d004e260ad46168f1321f718eef6a1 P 6b21299357f942dab7c574db37593c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.211004 14883 ts_tablet_manager.cc:1428] T 38244a7426a443efbd07ab8975ae0b75 P eae0557778094d6c9e874fb9f99ead31: Time spent starting tablet: real 0.008s user 0.006s sys 0.001s
I20250904 13:59:15.211844 14884 ts_tablet_manager.cc:1428] T 98d004e260ad46168f1321f718eef6a1 P 6b21299357f942dab7c574db37593c3b: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20250904 13:59:15.212169 14883 tablet_bootstrap.cc:492] T c69c4a3a981847d8a5febe6a619cc8a1 P eae0557778094d6c9e874fb9f99ead31: Bootstrap starting.
I20250904 13:59:15.213220 14884 tablet_bootstrap.cc:492] T 4903d9835fcc4652b77ae055c23b8a4d P 6b21299357f942dab7c574db37593c3b: Bootstrap starting.
I20250904 13:59:15.219583 14883 tablet_bootstrap.cc:654] T c69c4a3a981847d8a5febe6a619cc8a1 P eae0557778094d6c9e874fb9f99ead31: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.221364 14882 tablet_bootstrap.cc:492] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748: No bootstrap required, opened a new log
I20250904 13:59:15.221998 14884 tablet_bootstrap.cc:654] T 4903d9835fcc4652b77ae055c23b8a4d P 6b21299357f942dab7c574db37593c3b: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.222076 14882 ts_tablet_manager.cc:1397] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.011s
I20250904 13:59:15.225523 14882 raft_consensus.cc:357] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.226755 14882 raft_consensus.cc:383] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.227228 14882 raft_consensus.cc:738] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0cdd7953ea4648a402cfaaa28ce748, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.228426 14882 consensus_queue.cc:260] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.232187 14882 ts_tablet_manager.cc:1428] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20250904 13:59:15.233022 14883 tablet_bootstrap.cc:492] T c69c4a3a981847d8a5febe6a619cc8a1 P eae0557778094d6c9e874fb9f99ead31: No bootstrap required, opened a new log
I20250904 13:59:15.233485 14882 tablet_bootstrap.cc:492] T 77ea8776d09e48978e66a1c36c15cb56 P 1f0cdd7953ea4648a402cfaaa28ce748: Bootstrap starting.
I20250904 13:59:15.233723 14883 ts_tablet_manager.cc:1397] T c69c4a3a981847d8a5febe6a619cc8a1 P eae0557778094d6c9e874fb9f99ead31: Time spent bootstrapping tablet: real 0.022s user 0.017s sys 0.002s
I20250904 13:59:15.234045 14884 tablet_bootstrap.cc:492] T 4903d9835fcc4652b77ae055c23b8a4d P 6b21299357f942dab7c574db37593c3b: No bootstrap required, opened a new log
I20250904 13:59:15.234683 14884 ts_tablet_manager.cc:1397] T 4903d9835fcc4652b77ae055c23b8a4d P 6b21299357f942dab7c574db37593c3b: Time spent bootstrapping tablet: real 0.022s user 0.020s sys 0.000s
I20250904 13:59:15.237018 14883 raft_consensus.cc:357] T c69c4a3a981847d8a5febe6a619cc8a1 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.238055 14883 raft_consensus.cc:383] T c69c4a3a981847d8a5febe6a619cc8a1 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.237771 14884 raft_consensus.cc:357] T 4903d9835fcc4652b77ae055c23b8a4d P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.238507 14883 raft_consensus.cc:738] T c69c4a3a981847d8a5febe6a619cc8a1 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eae0557778094d6c9e874fb9f99ead31, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.238896 14884 raft_consensus.cc:383] T 4903d9835fcc4652b77ae055c23b8a4d P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.239342 14884 raft_consensus.cc:738] T 4903d9835fcc4652b77ae055c23b8a4d P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b21299357f942dab7c574db37593c3b, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.239571 14883 consensus_queue.cc:260] T c69c4a3a981847d8a5febe6a619cc8a1 P eae0557778094d6c9e874fb9f99ead31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.240235 14884 consensus_queue.cc:260] T 4903d9835fcc4652b77ae055c23b8a4d P 6b21299357f942dab7c574db37593c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.241267 14882 tablet_bootstrap.cc:654] T 77ea8776d09e48978e66a1c36c15cb56 P 1f0cdd7953ea4648a402cfaaa28ce748: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.242507 14883 ts_tablet_manager.cc:1428] T c69c4a3a981847d8a5febe6a619cc8a1 P eae0557778094d6c9e874fb9f99ead31: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20250904 13:59:15.243254 14884 ts_tablet_manager.cc:1428] T 4903d9835fcc4652b77ae055c23b8a4d P 6b21299357f942dab7c574db37593c3b: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20250904 13:59:15.243610 14883 tablet_bootstrap.cc:492] T 77ea8776d09e48978e66a1c36c15cb56 P eae0557778094d6c9e874fb9f99ead31: Bootstrap starting.
I20250904 13:59:15.244477 14884 tablet_bootstrap.cc:492] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b: Bootstrap starting.
I20250904 13:59:15.251060 14883 tablet_bootstrap.cc:654] T 77ea8776d09e48978e66a1c36c15cb56 P eae0557778094d6c9e874fb9f99ead31: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.253382 14884 tablet_bootstrap.cc:654] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.253445 14882 tablet_bootstrap.cc:492] T 77ea8776d09e48978e66a1c36c15cb56 P 1f0cdd7953ea4648a402cfaaa28ce748: No bootstrap required, opened a new log
I20250904 13:59:15.254362 14882 ts_tablet_manager.cc:1397] T 77ea8776d09e48978e66a1c36c15cb56 P 1f0cdd7953ea4648a402cfaaa28ce748: Time spent bootstrapping tablet: real 0.021s user 0.016s sys 0.003s
I20250904 13:59:15.257907 14882 raft_consensus.cc:357] T 77ea8776d09e48978e66a1c36c15cb56 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.259148 14882 raft_consensus.cc:383] T 77ea8776d09e48978e66a1c36c15cb56 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.259639 14882 raft_consensus.cc:738] T 77ea8776d09e48978e66a1c36c15cb56 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0cdd7953ea4648a402cfaaa28ce748, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.260828 14882 consensus_queue.cc:260] T 77ea8776d09e48978e66a1c36c15cb56 P 1f0cdd7953ea4648a402cfaaa28ce748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.263276 14883 tablet_bootstrap.cc:492] T 77ea8776d09e48978e66a1c36c15cb56 P eae0557778094d6c9e874fb9f99ead31: No bootstrap required, opened a new log
I20250904 13:59:15.263896 14883 ts_tablet_manager.cc:1397] T 77ea8776d09e48978e66a1c36c15cb56 P eae0557778094d6c9e874fb9f99ead31: Time spent bootstrapping tablet: real 0.021s user 0.015s sys 0.003s
I20250904 13:59:15.264501 14882 ts_tablet_manager.cc:1428] T 77ea8776d09e48978e66a1c36c15cb56 P 1f0cdd7953ea4648a402cfaaa28ce748: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20250904 13:59:15.265769 14882 tablet_bootstrap.cc:492] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748: Bootstrap starting.
I20250904 13:59:15.266410 14884 tablet_bootstrap.cc:492] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b: No bootstrap required, opened a new log
I20250904 13:59:15.267059 14884 ts_tablet_manager.cc:1397] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b: Time spent bootstrapping tablet: real 0.023s user 0.019s sys 0.002s
I20250904 13:59:15.267138 14883 raft_consensus.cc:357] T 77ea8776d09e48978e66a1c36c15cb56 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.268046 14883 raft_consensus.cc:383] T 77ea8776d09e48978e66a1c36c15cb56 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.268399 14883 raft_consensus.cc:738] T 77ea8776d09e48978e66a1c36c15cb56 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eae0557778094d6c9e874fb9f99ead31, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.269291 14883 consensus_queue.cc:260] T 77ea8776d09e48978e66a1c36c15cb56 P eae0557778094d6c9e874fb9f99ead31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.270180 14884 raft_consensus.cc:357] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.271441 14884 raft_consensus.cc:383] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.271823 14884 raft_consensus.cc:738] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b21299357f942dab7c574db37593c3b, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.272753 14884 consensus_queue.cc:260] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.273659 14882 tablet_bootstrap.cc:654] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.274502 14883 ts_tablet_manager.cc:1428] T 77ea8776d09e48978e66a1c36c15cb56 P eae0557778094d6c9e874fb9f99ead31: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20250904 13:59:15.275642 14883 tablet_bootstrap.cc:492] T 98d004e260ad46168f1321f718eef6a1 P eae0557778094d6c9e874fb9f99ead31: Bootstrap starting.
I20250904 13:59:15.276098 14884 ts_tablet_manager.cc:1428] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20250904 13:59:15.277643 14884 tablet_bootstrap.cc:492] T f1fb57f4c29f42f3901517c38d4fd2bd P 6b21299357f942dab7c574db37593c3b: Bootstrap starting.
I20250904 13:59:15.283535 14883 tablet_bootstrap.cc:654] T 98d004e260ad46168f1321f718eef6a1 P eae0557778094d6c9e874fb9f99ead31: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.285703 14884 tablet_bootstrap.cc:654] T f1fb57f4c29f42f3901517c38d4fd2bd P 6b21299357f942dab7c574db37593c3b: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.285781 14882 tablet_bootstrap.cc:492] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748: No bootstrap required, opened a new log
I20250904 13:59:15.287817 14882 ts_tablet_manager.cc:1397] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.015s
I20250904 13:59:15.291302 14882 raft_consensus.cc:357] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.292474 14882 raft_consensus.cc:383] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.292917 14882 raft_consensus.cc:738] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0cdd7953ea4648a402cfaaa28ce748, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.294060 14882 consensus_queue.cc:260] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.296267 14883 tablet_bootstrap.cc:492] T 98d004e260ad46168f1321f718eef6a1 P eae0557778094d6c9e874fb9f99ead31: No bootstrap required, opened a new log
I20250904 13:59:15.296785 14883 ts_tablet_manager.cc:1397] T 98d004e260ad46168f1321f718eef6a1 P eae0557778094d6c9e874fb9f99ead31: Time spent bootstrapping tablet: real 0.021s user 0.010s sys 0.007s
I20250904 13:59:15.297133 14882 ts_tablet_manager.cc:1428] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20250904 13:59:15.297561 14884 tablet_bootstrap.cc:492] T f1fb57f4c29f42f3901517c38d4fd2bd P 6b21299357f942dab7c574db37593c3b: No bootstrap required, opened a new log
I20250904 13:59:15.298053 14884 ts_tablet_manager.cc:1397] T f1fb57f4c29f42f3901517c38d4fd2bd P 6b21299357f942dab7c574db37593c3b: Time spent bootstrapping tablet: real 0.021s user 0.009s sys 0.008s
I20250904 13:59:15.298368 14882 tablet_bootstrap.cc:492] T 4eb02492ca4c4f0f990cb55611347087 P 1f0cdd7953ea4648a402cfaaa28ce748: Bootstrap starting.
I20250904 13:59:15.299722 14883 raft_consensus.cc:357] T 98d004e260ad46168f1321f718eef6a1 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.300516 14883 raft_consensus.cc:383] T 98d004e260ad46168f1321f718eef6a1 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.300844 14883 raft_consensus.cc:738] T 98d004e260ad46168f1321f718eef6a1 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eae0557778094d6c9e874fb9f99ead31, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.300630 14884 raft_consensus.cc:357] T f1fb57f4c29f42f3901517c38d4fd2bd P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.301375 14884 raft_consensus.cc:383] T f1fb57f4c29f42f3901517c38d4fd2bd P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.301704 14884 raft_consensus.cc:738] T f1fb57f4c29f42f3901517c38d4fd2bd P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b21299357f942dab7c574db37593c3b, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.301846 14883 consensus_queue.cc:260] T 98d004e260ad46168f1321f718eef6a1 P eae0557778094d6c9e874fb9f99ead31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.302683 14884 consensus_queue.cc:260] T f1fb57f4c29f42f3901517c38d4fd2bd P 6b21299357f942dab7c574db37593c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.304814 14883 ts_tablet_manager.cc:1428] T 98d004e260ad46168f1321f718eef6a1 P eae0557778094d6c9e874fb9f99ead31: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20250904 13:59:15.305267 14884 ts_tablet_manager.cc:1428] T f1fb57f4c29f42f3901517c38d4fd2bd P 6b21299357f942dab7c574db37593c3b: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20250904 13:59:15.305629 14882 tablet_bootstrap.cc:654] T 4eb02492ca4c4f0f990cb55611347087 P 1f0cdd7953ea4648a402cfaaa28ce748: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.305984 14883 tablet_bootstrap.cc:492] T 2746477d17a44175a64aed903c4cc706 P eae0557778094d6c9e874fb9f99ead31: Bootstrap starting.
I20250904 13:59:15.306260 14884 tablet_bootstrap.cc:492] T 38244a7426a443efbd07ab8975ae0b75 P 6b21299357f942dab7c574db37593c3b: Bootstrap starting.
I20250904 13:59:15.313216 14883 tablet_bootstrap.cc:654] T 2746477d17a44175a64aed903c4cc706 P eae0557778094d6c9e874fb9f99ead31: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.313616 14884 tablet_bootstrap.cc:654] T 38244a7426a443efbd07ab8975ae0b75 P 6b21299357f942dab7c574db37593c3b: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.318903 14882 tablet_bootstrap.cc:492] T 4eb02492ca4c4f0f990cb55611347087 P 1f0cdd7953ea4648a402cfaaa28ce748: No bootstrap required, opened a new log
I20250904 13:59:15.319734 14882 ts_tablet_manager.cc:1397] T 4eb02492ca4c4f0f990cb55611347087 P 1f0cdd7953ea4648a402cfaaa28ce748: Time spent bootstrapping tablet: real 0.022s user 0.012s sys 0.007s
I20250904 13:59:15.323338 14882 raft_consensus.cc:357] T 4eb02492ca4c4f0f990cb55611347087 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.324677 14882 raft_consensus.cc:383] T 4eb02492ca4c4f0f990cb55611347087 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.325127 14882 raft_consensus.cc:738] T 4eb02492ca4c4f0f990cb55611347087 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0cdd7953ea4648a402cfaaa28ce748, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.326344 14882 consensus_queue.cc:260] T 4eb02492ca4c4f0f990cb55611347087 P 1f0cdd7953ea4648a402cfaaa28ce748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.330049 14882 ts_tablet_manager.cc:1428] T 4eb02492ca4c4f0f990cb55611347087 P 1f0cdd7953ea4648a402cfaaa28ce748: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20250904 13:59:15.331537 14882 tablet_bootstrap.cc:492] T 4903d9835fcc4652b77ae055c23b8a4d P 1f0cdd7953ea4648a402cfaaa28ce748: Bootstrap starting.
I20250904 13:59:15.339725 14882 tablet_bootstrap.cc:654] T 4903d9835fcc4652b77ae055c23b8a4d P 1f0cdd7953ea4648a402cfaaa28ce748: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.353286 14883 tablet_bootstrap.cc:492] T 2746477d17a44175a64aed903c4cc706 P eae0557778094d6c9e874fb9f99ead31: No bootstrap required, opened a new log
I20250904 13:59:15.353516 14884 tablet_bootstrap.cc:492] T 38244a7426a443efbd07ab8975ae0b75 P 6b21299357f942dab7c574db37593c3b: No bootstrap required, opened a new log
I20250904 13:59:15.353852 14883 ts_tablet_manager.cc:1397] T 2746477d17a44175a64aed903c4cc706 P eae0557778094d6c9e874fb9f99ead31: Time spent bootstrapping tablet: real 0.048s user 0.029s sys 0.015s
I20250904 13:59:15.354216 14884 ts_tablet_manager.cc:1397] T 38244a7426a443efbd07ab8975ae0b75 P 6b21299357f942dab7c574db37593c3b: Time spent bootstrapping tablet: real 0.048s user 0.021s sys 0.015s
I20250904 13:59:15.357096 14883 raft_consensus.cc:357] T 2746477d17a44175a64aed903c4cc706 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.357853 14883 raft_consensus.cc:383] T 2746477d17a44175a64aed903c4cc706 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.358461 14883 raft_consensus.cc:738] T 2746477d17a44175a64aed903c4cc706 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eae0557778094d6c9e874fb9f99ead31, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.357822 14884 raft_consensus.cc:357] T 38244a7426a443efbd07ab8975ae0b75 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.358956 14884 raft_consensus.cc:383] T 38244a7426a443efbd07ab8975ae0b75 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.359266 14883 consensus_queue.cc:260] T 2746477d17a44175a64aed903c4cc706 P eae0557778094d6c9e874fb9f99ead31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.359270 14884 raft_consensus.cc:738] T 38244a7426a443efbd07ab8975ae0b75 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b21299357f942dab7c574db37593c3b, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.360929 14884 consensus_queue.cc:260] T 38244a7426a443efbd07ab8975ae0b75 P 6b21299357f942dab7c574db37593c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.361920 14883 ts_tablet_manager.cc:1428] T 2746477d17a44175a64aed903c4cc706 P eae0557778094d6c9e874fb9f99ead31: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20250904 13:59:15.362330 14882 tablet_bootstrap.cc:492] T 4903d9835fcc4652b77ae055c23b8a4d P 1f0cdd7953ea4648a402cfaaa28ce748: No bootstrap required, opened a new log
I20250904 13:59:15.362798 14882 ts_tablet_manager.cc:1397] T 4903d9835fcc4652b77ae055c23b8a4d P 1f0cdd7953ea4648a402cfaaa28ce748: Time spent bootstrapping tablet: real 0.032s user 0.012s sys 0.008s
I20250904 13:59:15.362950 14883 tablet_bootstrap.cc:492] T 68be2a2fe36345b4aaafe10bce4ab6cb P eae0557778094d6c9e874fb9f99ead31: Bootstrap starting.
I20250904 13:59:15.363831 14884 ts_tablet_manager.cc:1428] T 38244a7426a443efbd07ab8975ae0b75 P 6b21299357f942dab7c574db37593c3b: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20250904 13:59:15.364843 14884 tablet_bootstrap.cc:492] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b: Bootstrap starting.
I20250904 13:59:15.365696 14882 raft_consensus.cc:357] T 4903d9835fcc4652b77ae055c23b8a4d P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.366413 14882 raft_consensus.cc:383] T 4903d9835fcc4652b77ae055c23b8a4d P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.366679 14882 raft_consensus.cc:738] T 4903d9835fcc4652b77ae055c23b8a4d P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f0cdd7953ea4648a402cfaaa28ce748, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.367368 14882 consensus_queue.cc:260] T 4903d9835fcc4652b77ae055c23b8a4d P 1f0cdd7953ea4648a402cfaaa28ce748 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.369377 14883 tablet_bootstrap.cc:654] T 68be2a2fe36345b4aaafe10bce4ab6cb P eae0557778094d6c9e874fb9f99ead31: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.369716 14882 ts_tablet_manager.cc:1428] T 4903d9835fcc4652b77ae055c23b8a4d P 1f0cdd7953ea4648a402cfaaa28ce748: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20250904 13:59:15.373492 14884 tablet_bootstrap.cc:654] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.379303 14884 tablet_bootstrap.cc:492] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b: No bootstrap required, opened a new log
I20250904 13:59:15.379742 14884 ts_tablet_manager.cc:1397] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.000s
I20250904 13:59:15.381875 14884 raft_consensus.cc:357] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.382501 14884 raft_consensus.cc:383] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.382714 14884 raft_consensus.cc:738] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b21299357f942dab7c574db37593c3b, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.383322 14884 consensus_queue.cc:260] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.386150 14883 tablet_bootstrap.cc:492] T 68be2a2fe36345b4aaafe10bce4ab6cb P eae0557778094d6c9e874fb9f99ead31: No bootstrap required, opened a new log
I20250904 13:59:15.386695 14883 ts_tablet_manager.cc:1397] T 68be2a2fe36345b4aaafe10bce4ab6cb P eae0557778094d6c9e874fb9f99ead31: Time spent bootstrapping tablet: real 0.024s user 0.009s sys 0.008s
I20250904 13:59:15.388533 14884 ts_tablet_manager.cc:1428] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20250904 13:59:15.390874 14883 raft_consensus.cc:357] T 68be2a2fe36345b4aaafe10bce4ab6cb P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.391542 14883 raft_consensus.cc:383] T 68be2a2fe36345b4aaafe10bce4ab6cb P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.391762 14883 raft_consensus.cc:738] T 68be2a2fe36345b4aaafe10bce4ab6cb P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eae0557778094d6c9e874fb9f99ead31, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.392500 14883 consensus_queue.cc:260] T 68be2a2fe36345b4aaafe10bce4ab6cb P eae0557778094d6c9e874fb9f99ead31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.394809 14883 ts_tablet_manager.cc:1428] T 68be2a2fe36345b4aaafe10bce4ab6cb P eae0557778094d6c9e874fb9f99ead31: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20250904 13:59:15.395782 14883 tablet_bootstrap.cc:492] T f1fb57f4c29f42f3901517c38d4fd2bd P eae0557778094d6c9e874fb9f99ead31: Bootstrap starting.
I20250904 13:59:15.401539 14883 tablet_bootstrap.cc:654] T f1fb57f4c29f42f3901517c38d4fd2bd P eae0557778094d6c9e874fb9f99ead31: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:15.406009 14883 tablet_bootstrap.cc:492] T f1fb57f4c29f42f3901517c38d4fd2bd P eae0557778094d6c9e874fb9f99ead31: No bootstrap required, opened a new log
I20250904 13:59:15.406471 14883 ts_tablet_manager.cc:1397] T f1fb57f4c29f42f3901517c38d4fd2bd P eae0557778094d6c9e874fb9f99ead31: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20250904 13:59:15.408398 14883 raft_consensus.cc:357] T f1fb57f4c29f42f3901517c38d4fd2bd P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.408898 14883 raft_consensus.cc:383] T f1fb57f4c29f42f3901517c38d4fd2bd P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:15.409112 14883 raft_consensus.cc:738] T f1fb57f4c29f42f3901517c38d4fd2bd P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eae0557778094d6c9e874fb9f99ead31, State: Initialized, Role: FOLLOWER
I20250904 13:59:15.409669 14883 consensus_queue.cc:260] T f1fb57f4c29f42f3901517c38d4fd2bd P eae0557778094d6c9e874fb9f99ead31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:15.411265 14883 ts_tablet_manager.cc:1428] T f1fb57f4c29f42f3901517c38d4fd2bd P eae0557778094d6c9e874fb9f99ead31: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20250904 13:59:16.455075 14918 raft_consensus.cc:491] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:16.455578 14918 raft_consensus.cc:513] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.457842 14918 leader_election.cc:290] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f0cdd7953ea4648a402cfaaa28ce748 (127.10.98.129:43873), eae0557778094d6c9e874fb9f99ead31 (127.10.98.131:44951)
I20250904 13:59:16.472421 14687 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "77ea8776d09e48978e66a1c36c15cb56" candidate_uuid: "6b21299357f942dab7c574db37593c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" is_pre_election: true
I20250904 13:59:16.473371 14687 raft_consensus.cc:2466] T 77ea8776d09e48978e66a1c36c15cb56 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b21299357f942dab7c574db37593c3b in term 0.
I20250904 13:59:16.474642 14728 leader_election.cc:304] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0cdd7953ea4648a402cfaaa28ce748, 6b21299357f942dab7c574db37593c3b; no voters:
I20250904 13:59:16.474862 14836 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "77ea8776d09e48978e66a1c36c15cb56" candidate_uuid: "6b21299357f942dab7c574db37593c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae0557778094d6c9e874fb9f99ead31" is_pre_election: true
I20250904 13:59:16.475560 14918 raft_consensus.cc:2802] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:16.475556 14836 raft_consensus.cc:2466] T 77ea8776d09e48978e66a1c36c15cb56 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b21299357f942dab7c574db37593c3b in term 0.
I20250904 13:59:16.475951 14918 raft_consensus.cc:491] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:16.476298 14918 raft_consensus.cc:3058] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.481922 14918 raft_consensus.cc:513] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.483588 14918 leader_election.cc:290] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [CANDIDATE]: Term 1 election: Requested vote from peers 1f0cdd7953ea4648a402cfaaa28ce748 (127.10.98.129:43873), eae0557778094d6c9e874fb9f99ead31 (127.10.98.131:44951)
I20250904 13:59:16.484678 14836 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "77ea8776d09e48978e66a1c36c15cb56" candidate_uuid: "6b21299357f942dab7c574db37593c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae0557778094d6c9e874fb9f99ead31"
I20250904 13:59:16.484678 14687 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "77ea8776d09e48978e66a1c36c15cb56" candidate_uuid: "6b21299357f942dab7c574db37593c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0cdd7953ea4648a402cfaaa28ce748"
I20250904 13:59:16.485486 14836 raft_consensus.cc:3058] T 77ea8776d09e48978e66a1c36c15cb56 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.485491 14687 raft_consensus.cc:3058] T 77ea8776d09e48978e66a1c36c15cb56 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.491396 14687 raft_consensus.cc:2466] T 77ea8776d09e48978e66a1c36c15cb56 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b21299357f942dab7c574db37593c3b in term 1.
I20250904 13:59:16.491400 14836 raft_consensus.cc:2466] T 77ea8776d09e48978e66a1c36c15cb56 P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b21299357f942dab7c574db37593c3b in term 1.
I20250904 13:59:16.492578 14728 leader_election.cc:304] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0cdd7953ea4648a402cfaaa28ce748, 6b21299357f942dab7c574db37593c3b; no voters:
I20250904 13:59:16.493388 14918 raft_consensus.cc:2802] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:16.494601 14918 raft_consensus.cc:695] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [term 1 LEADER]: Becoming Leader. State: Replica: 6b21299357f942dab7c574db37593c3b, State: Running, Role: LEADER
I20250904 13:59:16.495316 14918 consensus_queue.cc:237] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.503612 14585 catalog_manager.cc:5582] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b21299357f942dab7c574db37593c3b (127.10.98.130). New cstate: current_term: 1 leader_uuid: "6b21299357f942dab7c574db37593c3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } health_report { overall_health: UNKNOWN } } }
I20250904 13:59:16.571084 14929 raft_consensus.cc:491] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:16.571568 14929 raft_consensus.cc:513] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.573906 14929 leader_election.cc:290] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b21299357f942dab7c574db37593c3b (127.10.98.130:42551), eae0557778094d6c9e874fb9f99ead31 (127.10.98.131:44951)
I20250904 13:59:16.586470 14836 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "98d004e260ad46168f1321f718eef6a1" candidate_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae0557778094d6c9e874fb9f99ead31" is_pre_election: true
I20250904 13:59:16.587070 14762 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "98d004e260ad46168f1321f718eef6a1" candidate_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b21299357f942dab7c574db37593c3b" is_pre_election: true
I20250904 13:59:16.587284 14836 raft_consensus.cc:2466] T 98d004e260ad46168f1321f718eef6a1 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f0cdd7953ea4648a402cfaaa28ce748 in term 0.
I20250904 13:59:16.587882 14762 raft_consensus.cc:2466] T 98d004e260ad46168f1321f718eef6a1 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f0cdd7953ea4648a402cfaaa28ce748 in term 0.
I20250904 13:59:16.588516 14651 leader_election.cc:304] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0cdd7953ea4648a402cfaaa28ce748, eae0557778094d6c9e874fb9f99ead31; no voters:
I20250904 13:59:16.589332 14929 raft_consensus.cc:2802] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:16.589726 14929 raft_consensus.cc:491] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:16.590039 14929 raft_consensus.cc:3058] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.595731 14929 raft_consensus.cc:513] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.597342 14929 leader_election.cc:290] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [CANDIDATE]: Term 1 election: Requested vote from peers 6b21299357f942dab7c574db37593c3b (127.10.98.130:42551), eae0557778094d6c9e874fb9f99ead31 (127.10.98.131:44951)
I20250904 13:59:16.598469 14836 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "98d004e260ad46168f1321f718eef6a1" candidate_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae0557778094d6c9e874fb9f99ead31"
I20250904 13:59:16.598465 14762 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "98d004e260ad46168f1321f718eef6a1" candidate_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b21299357f942dab7c574db37593c3b"
I20250904 13:59:16.599069 14836 raft_consensus.cc:3058] T 98d004e260ad46168f1321f718eef6a1 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.599135 14762 raft_consensus.cc:3058] T 98d004e260ad46168f1321f718eef6a1 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.605407 14836 raft_consensus.cc:2466] T 98d004e260ad46168f1321f718eef6a1 P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f0cdd7953ea4648a402cfaaa28ce748 in term 1.
I20250904 13:59:16.606092 14762 raft_consensus.cc:2466] T 98d004e260ad46168f1321f718eef6a1 P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f0cdd7953ea4648a402cfaaa28ce748 in term 1.
I20250904 13:59:16.606974 14651 leader_election.cc:304] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0cdd7953ea4648a402cfaaa28ce748, eae0557778094d6c9e874fb9f99ead31; no voters:
I20250904 13:59:16.607892 14929 raft_consensus.cc:2802] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:16.609479 14929 raft_consensus.cc:695] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 LEADER]: Becoming Leader. State: Replica: 1f0cdd7953ea4648a402cfaaa28ce748, State: Running, Role: LEADER
I20250904 13:59:16.610457 14929 consensus_queue.cc:237] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.612187 14933 raft_consensus.cc:491] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:16.612773 14933 raft_consensus.cc:513] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.615167 14933 leader_election.cc:290] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b21299357f942dab7c574db37593c3b (127.10.98.130:42551), eae0557778094d6c9e874fb9f99ead31 (127.10.98.131:44951)
I20250904 13:59:16.616112 14762 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a93701c10d644072a8889318fd08c99a" candidate_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b21299357f942dab7c574db37593c3b" is_pre_election: true
I20250904 13:59:16.616830 14762 raft_consensus.cc:2466] T a93701c10d644072a8889318fd08c99a P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f0cdd7953ea4648a402cfaaa28ce748 in term 0.
I20250904 13:59:16.617630 14836 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a93701c10d644072a8889318fd08c99a" candidate_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae0557778094d6c9e874fb9f99ead31" is_pre_election: true
I20250904 13:59:16.618368 14836 raft_consensus.cc:2466] T a93701c10d644072a8889318fd08c99a P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f0cdd7953ea4648a402cfaaa28ce748 in term 0.
I20250904 13:59:16.618060 14652 leader_election.cc:304] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0cdd7953ea4648a402cfaaa28ce748, 6b21299357f942dab7c574db37593c3b; no voters:
I20250904 13:59:16.620090 14935 raft_consensus.cc:491] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:16.620149 14933 raft_consensus.cc:2802] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:16.620651 14933 raft_consensus.cc:491] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:16.620616 14935 raft_consensus.cc:513] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.621093 14933 raft_consensus.cc:3058] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.621402 14585 catalog_manager.cc:5582] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f0cdd7953ea4648a402cfaaa28ce748 (127.10.98.129). New cstate: current_term: 1 leader_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } health_report { overall_health: UNKNOWN } } }
I20250904 13:59:16.628969 14935 leader_election.cc:290] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b21299357f942dab7c574db37593c3b (127.10.98.130:42551), 1f0cdd7953ea4648a402cfaaa28ce748 (127.10.98.129:43873)
I20250904 13:59:16.632725 14933 raft_consensus.cc:513] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.635658 14933 leader_election.cc:290] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [CANDIDATE]: Term 1 election: Requested vote from peers 6b21299357f942dab7c574db37593c3b (127.10.98.130:42551), eae0557778094d6c9e874fb9f99ead31 (127.10.98.131:44951)
I20250904 13:59:16.635918 14762 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a93701c10d644072a8889318fd08c99a" candidate_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b21299357f942dab7c574db37593c3b"
I20250904 13:59:16.636605 14762 raft_consensus.cc:3058] T a93701c10d644072a8889318fd08c99a P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.637006 14836 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "a93701c10d644072a8889318fd08c99a" candidate_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae0557778094d6c9e874fb9f99ead31"
I20250904 13:59:16.637629 14836 raft_consensus.cc:3058] T a93701c10d644072a8889318fd08c99a P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.645000 14762 raft_consensus.cc:2466] T a93701c10d644072a8889318fd08c99a P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f0cdd7953ea4648a402cfaaa28ce748 in term 1.
I20250904 13:59:16.645468 14836 raft_consensus.cc:2466] T a93701c10d644072a8889318fd08c99a P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f0cdd7953ea4648a402cfaaa28ce748 in term 1.
I20250904 13:59:16.646337 14652 leader_election.cc:304] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0cdd7953ea4648a402cfaaa28ce748, 6b21299357f942dab7c574db37593c3b; no voters:
I20250904 13:59:16.647611 14933 raft_consensus.cc:2802] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:16.647739 14762 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4903d9835fcc4652b77ae055c23b8a4d" candidate_uuid: "eae0557778094d6c9e874fb9f99ead31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b21299357f942dab7c574db37593c3b" is_pre_election: true
I20250904 13:59:16.648152 14933 raft_consensus.cc:695] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 LEADER]: Becoming Leader. State: Replica: 1f0cdd7953ea4648a402cfaaa28ce748, State: Running, Role: LEADER
I20250904 13:59:16.648300 14687 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4903d9835fcc4652b77ae055c23b8a4d" candidate_uuid: "eae0557778094d6c9e874fb9f99ead31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" is_pre_election: true
I20250904 13:59:16.648427 14762 raft_consensus.cc:2466] T 4903d9835fcc4652b77ae055c23b8a4d P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eae0557778094d6c9e874fb9f99ead31 in term 0.
I20250904 13:59:16.648950 14687 raft_consensus.cc:2466] T 4903d9835fcc4652b77ae055c23b8a4d P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eae0557778094d6c9e874fb9f99ead31 in term 0.
I20250904 13:59:16.649096 14933 consensus_queue.cc:237] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.649704 14801 leader_election.cc:304] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b21299357f942dab7c574db37593c3b, eae0557778094d6c9e874fb9f99ead31; no voters:
I20250904 13:59:16.650681 14935 raft_consensus.cc:2802] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:16.651007 14935 raft_consensus.cc:491] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:16.651223 14935 raft_consensus.cc:3058] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.657719 14585 catalog_manager.cc:5582] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f0cdd7953ea4648a402cfaaa28ce748 (127.10.98.129). New cstate: current_term: 1 leader_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } health_report { overall_health: UNKNOWN } } }
I20250904 13:59:16.658437 14935 raft_consensus.cc:513] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.661118 14762 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4903d9835fcc4652b77ae055c23b8a4d" candidate_uuid: "eae0557778094d6c9e874fb9f99ead31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b21299357f942dab7c574db37593c3b"
I20250904 13:59:16.661258 14687 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4903d9835fcc4652b77ae055c23b8a4d" candidate_uuid: "eae0557778094d6c9e874fb9f99ead31" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0cdd7953ea4648a402cfaaa28ce748"
I20250904 13:59:16.661643 14762 raft_consensus.cc:3058] T 4903d9835fcc4652b77ae055c23b8a4d P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.661897 14687 raft_consensus.cc:3058] T 4903d9835fcc4652b77ae055c23b8a4d P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.660203 14935 leader_election.cc:290] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [CANDIDATE]: Term 1 election: Requested vote from peers 6b21299357f942dab7c574db37593c3b (127.10.98.130:42551), 1f0cdd7953ea4648a402cfaaa28ce748 (127.10.98.129:43873)
I20250904 13:59:16.668563 14687 raft_consensus.cc:2466] T 4903d9835fcc4652b77ae055c23b8a4d P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eae0557778094d6c9e874fb9f99ead31 in term 1.
I20250904 13:59:16.669646 14802 leader_election.cc:304] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0cdd7953ea4648a402cfaaa28ce748, eae0557778094d6c9e874fb9f99ead31; no voters:
I20250904 13:59:16.670450 14935 raft_consensus.cc:2802] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:16.670562 14762 raft_consensus.cc:2466] T 4903d9835fcc4652b77ae055c23b8a4d P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eae0557778094d6c9e874fb9f99ead31 in term 1.
I20250904 13:59:16.671844 14935 raft_consensus.cc:695] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [term 1 LEADER]: Becoming Leader. State: Replica: eae0557778094d6c9e874fb9f99ead31, State: Running, Role: LEADER
I20250904 13:59:16.672732 14935 consensus_queue.cc:237] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.679976 14585 catalog_manager.cc:5582] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 reported cstate change: term changed from 0 to 1, leader changed from <none> to eae0557778094d6c9e874fb9f99ead31 (127.10.98.131). New cstate: current_term: 1 leader_uuid: "eae0557778094d6c9e874fb9f99ead31" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } health_report { overall_health: HEALTHY } } }
I20250904 13:59:16.783543 14918 raft_consensus.cc:491] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:16.784206 14918 raft_consensus.cc:513] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.786001 14918 leader_election.cc:290] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f0cdd7953ea4648a402cfaaa28ce748 (127.10.98.129:43873), eae0557778094d6c9e874fb9f99ead31 (127.10.98.131:44951)
I20250904 13:59:16.786955 14687 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c69c4a3a981847d8a5febe6a619cc8a1" candidate_uuid: "6b21299357f942dab7c574db37593c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" is_pre_election: true
I20250904 13:59:16.787166 14836 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c69c4a3a981847d8a5febe6a619cc8a1" candidate_uuid: "6b21299357f942dab7c574db37593c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae0557778094d6c9e874fb9f99ead31" is_pre_election: true
I20250904 13:59:16.787580 14687 raft_consensus.cc:2466] T c69c4a3a981847d8a5febe6a619cc8a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b21299357f942dab7c574db37593c3b in term 0.
I20250904 13:59:16.787760 14836 raft_consensus.cc:2466] T c69c4a3a981847d8a5febe6a619cc8a1 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b21299357f942dab7c574db37593c3b in term 0.
I20250904 13:59:16.788558 14728 leader_election.cc:304] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0cdd7953ea4648a402cfaaa28ce748, 6b21299357f942dab7c574db37593c3b; no voters:
I20250904 13:59:16.789247 14918 raft_consensus.cc:2802] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:16.789613 14918 raft_consensus.cc:491] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:16.789877 14918 raft_consensus.cc:3058] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.795571 14918 raft_consensus.cc:513] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.797300 14918 leader_election.cc:290] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [CANDIDATE]: Term 1 election: Requested vote from peers 1f0cdd7953ea4648a402cfaaa28ce748 (127.10.98.129:43873), eae0557778094d6c9e874fb9f99ead31 (127.10.98.131:44951)
I20250904 13:59:16.798381 14836 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c69c4a3a981847d8a5febe6a619cc8a1" candidate_uuid: "6b21299357f942dab7c574db37593c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae0557778094d6c9e874fb9f99ead31"
I20250904 13:59:16.798336 14687 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "c69c4a3a981847d8a5febe6a619cc8a1" candidate_uuid: "6b21299357f942dab7c574db37593c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0cdd7953ea4648a402cfaaa28ce748"
I20250904 13:59:16.799034 14836 raft_consensus.cc:3058] T c69c4a3a981847d8a5febe6a619cc8a1 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.799088 14687 raft_consensus.cc:3058] T c69c4a3a981847d8a5febe6a619cc8a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.804529 14687 raft_consensus.cc:2466] T c69c4a3a981847d8a5febe6a619cc8a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b21299357f942dab7c574db37593c3b in term 1.
I20250904 13:59:16.804585 14836 raft_consensus.cc:2466] T c69c4a3a981847d8a5febe6a619cc8a1 P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b21299357f942dab7c574db37593c3b in term 1.
I20250904 13:59:16.805753 14728 leader_election.cc:304] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0cdd7953ea4648a402cfaaa28ce748, 6b21299357f942dab7c574db37593c3b; no voters:
I20250904 13:59:16.806497 14918 raft_consensus.cc:2802] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:16.806946 14918 raft_consensus.cc:695] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [term 1 LEADER]: Becoming Leader. State: Replica: 6b21299357f942dab7c574db37593c3b, State: Running, Role: LEADER
I20250904 13:59:16.807610 14918 consensus_queue.cc:237] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.815696 14585 catalog_manager.cc:5582] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b21299357f942dab7c574db37593c3b (127.10.98.130). New cstate: current_term: 1 leader_uuid: "6b21299357f942dab7c574db37593c3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } health_report { overall_health: UNKNOWN } } }
I20250904 13:59:16.886488 14918 raft_consensus.cc:491] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:16.887017 14918 raft_consensus.cc:513] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.888906 14918 leader_election.cc:290] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f0cdd7953ea4648a402cfaaa28ce748 (127.10.98.129:43873), eae0557778094d6c9e874fb9f99ead31 (127.10.98.131:44951)
I20250904 13:59:16.889808 14687 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "68be2a2fe36345b4aaafe10bce4ab6cb" candidate_uuid: "6b21299357f942dab7c574db37593c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" is_pre_election: true
I20250904 13:59:16.889940 14836 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "68be2a2fe36345b4aaafe10bce4ab6cb" candidate_uuid: "6b21299357f942dab7c574db37593c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae0557778094d6c9e874fb9f99ead31" is_pre_election: true
I20250904 13:59:16.890669 14687 raft_consensus.cc:2466] T 68be2a2fe36345b4aaafe10bce4ab6cb P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b21299357f942dab7c574db37593c3b in term 0.
I20250904 13:59:16.890689 14836 raft_consensus.cc:2466] T 68be2a2fe36345b4aaafe10bce4ab6cb P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b21299357f942dab7c574db37593c3b in term 0.
I20250904 13:59:16.892002 14726 leader_election.cc:304] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b21299357f942dab7c574db37593c3b, eae0557778094d6c9e874fb9f99ead31; no voters:
I20250904 13:59:16.892741 14918 raft_consensus.cc:2802] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:16.893095 14918 raft_consensus.cc:491] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:16.893327 14918 raft_consensus.cc:3058] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.899047 14918 raft_consensus.cc:513] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.900727 14918 leader_election.cc:290] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [CANDIDATE]: Term 1 election: Requested vote from peers 1f0cdd7953ea4648a402cfaaa28ce748 (127.10.98.129:43873), eae0557778094d6c9e874fb9f99ead31 (127.10.98.131:44951)
I20250904 13:59:16.901695 14687 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "68be2a2fe36345b4aaafe10bce4ab6cb" candidate_uuid: "6b21299357f942dab7c574db37593c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0cdd7953ea4648a402cfaaa28ce748"
I20250904 13:59:16.901955 14836 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "68be2a2fe36345b4aaafe10bce4ab6cb" candidate_uuid: "6b21299357f942dab7c574db37593c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae0557778094d6c9e874fb9f99ead31"
I20250904 13:59:16.902463 14687 raft_consensus.cc:3058] T 68be2a2fe36345b4aaafe10bce4ab6cb P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.902624 14836 raft_consensus.cc:3058] T 68be2a2fe36345b4aaafe10bce4ab6cb P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.909152 14687 raft_consensus.cc:2466] T 68be2a2fe36345b4aaafe10bce4ab6cb P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b21299357f942dab7c574db37593c3b in term 1.
I20250904 13:59:16.909154 14836 raft_consensus.cc:2466] T 68be2a2fe36345b4aaafe10bce4ab6cb P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b21299357f942dab7c574db37593c3b in term 1.
I20250904 13:59:16.909905 14918 raft_consensus.cc:491] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:16.910758 14728 leader_election.cc:304] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1f0cdd7953ea4648a402cfaaa28ce748, 6b21299357f942dab7c574db37593c3b, eae0557778094d6c9e874fb9f99ead31; no voters:
I20250904 13:59:16.910537 14918 raft_consensus.cc:513] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.911700 14923 raft_consensus.cc:2802] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:16.912611 14918 leader_election.cc:290] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f0cdd7953ea4648a402cfaaa28ce748 (127.10.98.129:43873), eae0557778094d6c9e874fb9f99ead31 (127.10.98.131:44951)
I20250904 13:59:16.914028 14923 raft_consensus.cc:695] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [term 1 LEADER]: Becoming Leader. State: Replica: 6b21299357f942dab7c574db37593c3b, State: Running, Role: LEADER
I20250904 13:59:16.914222 14918 raft_consensus.cc:491] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:16.913813 14687 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4eb02492ca4c4f0f990cb55611347087" candidate_uuid: "6b21299357f942dab7c574db37593c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" is_pre_election: true
I20250904 13:59:16.914747 14836 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4eb02492ca4c4f0f990cb55611347087" candidate_uuid: "6b21299357f942dab7c574db37593c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae0557778094d6c9e874fb9f99ead31" is_pre_election: true
I20250904 13:59:16.914870 14918 raft_consensus.cc:513] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.915315 14687 raft_consensus.cc:2466] T 4eb02492ca4c4f0f990cb55611347087 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b21299357f942dab7c574db37593c3b in term 0.
I20250904 13:59:16.915120 14923 consensus_queue.cc:237] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.915571 14836 raft_consensus.cc:2466] T 4eb02492ca4c4f0f990cb55611347087 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b21299357f942dab7c574db37593c3b in term 0.
I20250904 13:59:16.916782 14728 leader_election.cc:304] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0cdd7953ea4648a402cfaaa28ce748, 6b21299357f942dab7c574db37593c3b; no voters:
I20250904 13:59:16.917836 14942 raft_consensus.cc:2802] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:16.918004 14918 leader_election.cc:290] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f0cdd7953ea4648a402cfaaa28ce748 (127.10.98.129:43873), eae0557778094d6c9e874fb9f99ead31 (127.10.98.131:44951)
I20250904 13:59:16.918368 14942 raft_consensus.cc:491] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:16.918901 14942 raft_consensus.cc:3058] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.919214 14687 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2746477d17a44175a64aed903c4cc706" candidate_uuid: "6b21299357f942dab7c574db37593c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" is_pre_election: true
I20250904 13:59:16.919864 14687 raft_consensus.cc:2466] T 2746477d17a44175a64aed903c4cc706 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b21299357f942dab7c574db37593c3b in term 0.
I20250904 13:59:16.920842 14728 leader_election.cc:304] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0cdd7953ea4648a402cfaaa28ce748, 6b21299357f942dab7c574db37593c3b; no voters:
I20250904 13:59:16.921864 14918 raft_consensus.cc:2802] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:16.922407 14918 raft_consensus.cc:491] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:16.922813 14918 raft_consensus.cc:3058] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.926831 14836 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2746477d17a44175a64aed903c4cc706" candidate_uuid: "6b21299357f942dab7c574db37593c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae0557778094d6c9e874fb9f99ead31" is_pre_election: true
I20250904 13:59:16.927800 14836 raft_consensus.cc:2466] T 2746477d17a44175a64aed903c4cc706 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b21299357f942dab7c574db37593c3b in term 0.
I20250904 13:59:16.927278 14585 catalog_manager.cc:5582] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b21299357f942dab7c574db37593c3b (127.10.98.130). New cstate: current_term: 1 leader_uuid: "6b21299357f942dab7c574db37593c3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } health_report { overall_health: UNKNOWN } } }
I20250904 13:59:16.928006 14942 raft_consensus.cc:513] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.930284 14942 leader_election.cc:290] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b [CANDIDATE]: Term 1 election: Requested vote from peers 1f0cdd7953ea4648a402cfaaa28ce748 (127.10.98.129:43873), eae0557778094d6c9e874fb9f99ead31 (127.10.98.131:44951)
I20250904 13:59:16.930596 14918 raft_consensus.cc:513] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.931264 14687 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4eb02492ca4c4f0f990cb55611347087" candidate_uuid: "6b21299357f942dab7c574db37593c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0cdd7953ea4648a402cfaaa28ce748"
I20250904 13:59:16.931931 14687 raft_consensus.cc:3058] T 4eb02492ca4c4f0f990cb55611347087 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.932608 14836 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4eb02492ca4c4f0f990cb55611347087" candidate_uuid: "6b21299357f942dab7c574db37593c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae0557778094d6c9e874fb9f99ead31"
I20250904 13:59:16.933331 14836 raft_consensus.cc:3058] T 4eb02492ca4c4f0f990cb55611347087 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.933907 14686 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2746477d17a44175a64aed903c4cc706" candidate_uuid: "6b21299357f942dab7c574db37593c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f0cdd7953ea4648a402cfaaa28ce748"
I20250904 13:59:16.934625 14686 raft_consensus.cc:3058] T 2746477d17a44175a64aed903c4cc706 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.939766 14687 raft_consensus.cc:2466] T 4eb02492ca4c4f0f990cb55611347087 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b21299357f942dab7c574db37593c3b in term 1.
I20250904 13:59:16.940907 14728 leader_election.cc:304] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0cdd7953ea4648a402cfaaa28ce748, 6b21299357f942dab7c574db37593c3b; no voters:
I20250904 13:59:16.941635 14942 raft_consensus.cc:2802] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:16.942060 14942 raft_consensus.cc:695] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b [term 1 LEADER]: Becoming Leader. State: Replica: 6b21299357f942dab7c574db37593c3b, State: Running, Role: LEADER
I20250904 13:59:16.942029 14836 raft_consensus.cc:2466] T 4eb02492ca4c4f0f990cb55611347087 P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b21299357f942dab7c574db37593c3b in term 1.
I20250904 13:59:16.943184 14918 leader_election.cc:290] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [CANDIDATE]: Term 1 election: Requested vote from peers 1f0cdd7953ea4648a402cfaaa28ce748 (127.10.98.129:43873), eae0557778094d6c9e874fb9f99ead31 (127.10.98.131:44951)
I20250904 13:59:16.943375 14686 raft_consensus.cc:2466] T 2746477d17a44175a64aed903c4cc706 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b21299357f942dab7c574db37593c3b in term 1.
I20250904 13:59:16.943209 14942 consensus_queue.cc:237] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.944641 14835 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "2746477d17a44175a64aed903c4cc706" candidate_uuid: "6b21299357f942dab7c574db37593c3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae0557778094d6c9e874fb9f99ead31"
I20250904 13:59:16.945169 14728 leader_election.cc:304] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0cdd7953ea4648a402cfaaa28ce748, 6b21299357f942dab7c574db37593c3b; no voters:
I20250904 13:59:16.945359 14835 raft_consensus.cc:3058] T 2746477d17a44175a64aed903c4cc706 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:16.946067 14918 raft_consensus.cc:2802] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:16.946628 14918 raft_consensus.cc:695] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [term 1 LEADER]: Becoming Leader. State: Replica: 6b21299357f942dab7c574db37593c3b, State: Running, Role: LEADER
I20250904 13:59:16.947655 14918 consensus_queue.cc:237] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:16.952805 14585 catalog_manager.cc:5582] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b21299357f942dab7c574db37593c3b (127.10.98.130). New cstate: current_term: 1 leader_uuid: "6b21299357f942dab7c574db37593c3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } health_report { overall_health: UNKNOWN } } }
I20250904 13:59:16.953851 14835 raft_consensus.cc:2466] T 2746477d17a44175a64aed903c4cc706 P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b21299357f942dab7c574db37593c3b in term 1.
I20250904 13:59:16.965351 14585 catalog_manager.cc:5582] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b21299357f942dab7c574db37593c3b (127.10.98.130). New cstate: current_term: 1 leader_uuid: "6b21299357f942dab7c574db37593c3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } health_report { overall_health: UNKNOWN } } }
I20250904 13:59:16.973062 14918 consensus_queue.cc:1035] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250904 13:59:16.985194 14942 consensus_queue.cc:1035] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250904 13:59:16.998651 14933 consensus_queue.cc:1035] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:59:17.010509 14933 consensus_queue.cc:1035] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:59:17.024291 14929 raft_consensus.cc:491] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:17.024822 14929 raft_consensus.cc:513] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:17.026523 14929 leader_election.cc:290] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b21299357f942dab7c574db37593c3b (127.10.98.130:42551), eae0557778094d6c9e874fb9f99ead31 (127.10.98.131:44951)
I20250904 13:59:17.027515 14762 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "38244a7426a443efbd07ab8975ae0b75" candidate_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b21299357f942dab7c574db37593c3b" is_pre_election: true
I20250904 13:59:17.027733 14835 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "38244a7426a443efbd07ab8975ae0b75" candidate_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae0557778094d6c9e874fb9f99ead31" is_pre_election: true
I20250904 13:59:17.028157 14762 raft_consensus.cc:2466] T 38244a7426a443efbd07ab8975ae0b75 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f0cdd7953ea4648a402cfaaa28ce748 in term 0.
I20250904 13:59:17.028415 14835 raft_consensus.cc:2466] T 38244a7426a443efbd07ab8975ae0b75 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f0cdd7953ea4648a402cfaaa28ce748 in term 0.
I20250904 13:59:17.029306 14652 leader_election.cc:304] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0cdd7953ea4648a402cfaaa28ce748, 6b21299357f942dab7c574db37593c3b; no voters:
I20250904 13:59:17.030064 14929 raft_consensus.cc:2802] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:17.030478 14929 raft_consensus.cc:491] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:17.030833 14929 raft_consensus.cc:3058] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:17.033504 14933 consensus_queue.cc:1035] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:59:17.037806 14929 raft_consensus.cc:513] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:17.040825 14762 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "38244a7426a443efbd07ab8975ae0b75" candidate_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b21299357f942dab7c574db37593c3b"
I20250904 13:59:17.041429 14929 leader_election.cc:290] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [CANDIDATE]: Term 1 election: Requested vote from peers 6b21299357f942dab7c574db37593c3b (127.10.98.130:42551), eae0557778094d6c9e874fb9f99ead31 (127.10.98.131:44951)
I20250904 13:59:17.042737 14835 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "38244a7426a443efbd07ab8975ae0b75" candidate_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae0557778094d6c9e874fb9f99ead31"
I20250904 13:59:17.043351 14835 raft_consensus.cc:3058] T 38244a7426a443efbd07ab8975ae0b75 P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:17.051479 14835 raft_consensus.cc:2466] T 38244a7426a443efbd07ab8975ae0b75 P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f0cdd7953ea4648a402cfaaa28ce748 in term 1.
I20250904 13:59:17.041448 14762 raft_consensus.cc:3058] T 38244a7426a443efbd07ab8975ae0b75 P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:17.053033 14651 leader_election.cc:304] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0cdd7953ea4648a402cfaaa28ce748, eae0557778094d6c9e874fb9f99ead31; no voters:
I20250904 13:59:17.052881 14929 consensus_queue.cc:1035] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:59:17.053900 14952 raft_consensus.cc:2802] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:17.054417 14952 raft_consensus.cc:695] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 LEADER]: Becoming Leader. State: Replica: 1f0cdd7953ea4648a402cfaaa28ce748, State: Running, Role: LEADER
I20250904 13:59:17.055303 14952 consensus_queue.cc:237] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:17.061477 14762 raft_consensus.cc:2466] T 38244a7426a443efbd07ab8975ae0b75 P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f0cdd7953ea4648a402cfaaa28ce748 in term 1.
I20250904 13:59:17.070626 14584 catalog_manager.cc:5582] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f0cdd7953ea4648a402cfaaa28ce748 (127.10.98.129). New cstate: current_term: 1 leader_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } health_report { overall_health: UNKNOWN } } }
I20250904 13:59:17.145008 14952 raft_consensus.cc:491] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:17.145741 14952 raft_consensus.cc:513] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:17.148319 14952 leader_election.cc:290] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b21299357f942dab7c574db37593c3b (127.10.98.130:42551), eae0557778094d6c9e874fb9f99ead31 (127.10.98.131:44951)
I20250904 13:59:17.149211 14762 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f1fb57f4c29f42f3901517c38d4fd2bd" candidate_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b21299357f942dab7c574db37593c3b" is_pre_election: true
I20250904 13:59:17.149364 14836 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f1fb57f4c29f42f3901517c38d4fd2bd" candidate_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae0557778094d6c9e874fb9f99ead31" is_pre_election: true
I20250904 13:59:17.149796 14762 raft_consensus.cc:2466] T f1fb57f4c29f42f3901517c38d4fd2bd P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f0cdd7953ea4648a402cfaaa28ce748 in term 0.
I20250904 13:59:17.149955 14836 raft_consensus.cc:2466] T f1fb57f4c29f42f3901517c38d4fd2bd P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f0cdd7953ea4648a402cfaaa28ce748 in term 0.
I20250904 13:59:17.150839 14652 leader_election.cc:304] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0cdd7953ea4648a402cfaaa28ce748, 6b21299357f942dab7c574db37593c3b; no voters:
I20250904 13:59:17.151568 14952 raft_consensus.cc:2802] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:17.151930 14952 raft_consensus.cc:491] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:17.152292 14952 raft_consensus.cc:3058] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:17.158274 14952 raft_consensus.cc:513] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:17.160013 14952 leader_election.cc:290] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 [CANDIDATE]: Term 1 election: Requested vote from peers 6b21299357f942dab7c574db37593c3b (127.10.98.130:42551), eae0557778094d6c9e874fb9f99ead31 (127.10.98.131:44951)
I20250904 13:59:17.160874 14762 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f1fb57f4c29f42f3901517c38d4fd2bd" candidate_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b21299357f942dab7c574db37593c3b"
I20250904 13:59:17.161079 14836 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "f1fb57f4c29f42f3901517c38d4fd2bd" candidate_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eae0557778094d6c9e874fb9f99ead31"
I20250904 13:59:17.161403 14762 raft_consensus.cc:3058] T f1fb57f4c29f42f3901517c38d4fd2bd P 6b21299357f942dab7c574db37593c3b [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:17.161592 14836 raft_consensus.cc:3058] T f1fb57f4c29f42f3901517c38d4fd2bd P eae0557778094d6c9e874fb9f99ead31 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:17.167135 14836 raft_consensus.cc:2466] T f1fb57f4c29f42f3901517c38d4fd2bd P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f0cdd7953ea4648a402cfaaa28ce748 in term 1.
I20250904 13:59:17.167135 14762 raft_consensus.cc:2466] T f1fb57f4c29f42f3901517c38d4fd2bd P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f0cdd7953ea4648a402cfaaa28ce748 in term 1.
I20250904 13:59:17.168391 14652 leader_election.cc:304] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f0cdd7953ea4648a402cfaaa28ce748, 6b21299357f942dab7c574db37593c3b; no voters:
I20250904 13:59:17.169127 14952 raft_consensus.cc:2802] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:17.169487 14952 raft_consensus.cc:695] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 LEADER]: Becoming Leader. State: Replica: 1f0cdd7953ea4648a402cfaaa28ce748, State: Running, Role: LEADER
I20250904 13:59:17.170356 14952 consensus_queue.cc:237] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } }
I20250904 13:59:17.179177 14584 catalog_manager.cc:5582] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f0cdd7953ea4648a402cfaaa28ce748 (127.10.98.129). New cstate: current_term: 1 leader_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 } health_report { overall_health: UNKNOWN } } }
I20250904 13:59:17.208577 14935 consensus_queue.cc:1035] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:59:17.221168 14960 consensus_queue.cc:1035] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250904 13:59:17.273135 14942 consensus_queue.cc:1035] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:59:17.277091 14923 consensus_queue.cc:1035] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:59:17.318579 14923 consensus_queue.cc:1035] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:59:17.329263 14923 consensus_queue.cc:1035] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:59:17.401551 14923 consensus_queue.cc:1035] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:59:17.412854 14923 consensus_queue.cc:1035] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f0cdd7953ea4648a402cfaaa28ce748" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 43873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:59:17.428932 10634 tablet_server.cc:178] TabletServer@127.10.98.129:0 shutting down...
I20250904 13:59:17.439726 14923 consensus_queue.cc:1035] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20250904 13:59:17.453378 14728 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20250904 13:59:17.459336 14728 consensus_peers.cc:489] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b -> Peer 1f0cdd7953ea4648a402cfaaa28ce748 (127.10.98.129:43873): Couldn't send request to peer 1f0cdd7953ea4648a402cfaaa28ce748. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250904 13:59:17.470562 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:59:17.471429 10634 tablet_replica.cc:331] T 4903d9835fcc4652b77ae055c23b8a4d P 1f0cdd7953ea4648a402cfaaa28ce748: stopping tablet replica
I20250904 13:59:17.472098 10634 raft_consensus.cc:2241] T 4903d9835fcc4652b77ae055c23b8a4d P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:17.472569 10634 raft_consensus.cc:2270] T 4903d9835fcc4652b77ae055c23b8a4d P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.475241 10634 tablet_replica.cc:331] T 4eb02492ca4c4f0f990cb55611347087 P 1f0cdd7953ea4648a402cfaaa28ce748: stopping tablet replica
I20250904 13:59:17.475888 10634 raft_consensus.cc:2241] T 4eb02492ca4c4f0f990cb55611347087 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:17.476285 10634 raft_consensus.cc:2270] T 4eb02492ca4c4f0f990cb55611347087 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.478600 10634 tablet_replica.cc:331] T 77ea8776d09e48978e66a1c36c15cb56 P 1f0cdd7953ea4648a402cfaaa28ce748: stopping tablet replica
I20250904 13:59:17.479218 10634 raft_consensus.cc:2241] T 77ea8776d09e48978e66a1c36c15cb56 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:17.479775 10634 raft_consensus.cc:2270] T 77ea8776d09e48978e66a1c36c15cb56 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.479945 14933 consensus_queue.cc:1035] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b21299357f942dab7c574db37593c3b" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 42551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:59:17.482707 10634 tablet_replica.cc:331] T c69c4a3a981847d8a5febe6a619cc8a1 P 1f0cdd7953ea4648a402cfaaa28ce748: stopping tablet replica
I20250904 13:59:17.483435 10634 raft_consensus.cc:2241] T c69c4a3a981847d8a5febe6a619cc8a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:17.483896 10634 raft_consensus.cc:2270] T c69c4a3a981847d8a5febe6a619cc8a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.487020 10634 tablet_replica.cc:331] T 68be2a2fe36345b4aaafe10bce4ab6cb P 1f0cdd7953ea4648a402cfaaa28ce748: stopping tablet replica
I20250904 13:59:17.487834 10634 raft_consensus.cc:2241] T 68be2a2fe36345b4aaafe10bce4ab6cb P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:17.488982 10634 raft_consensus.cc:2270] T 68be2a2fe36345b4aaafe10bce4ab6cb P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.492475 10634 tablet_replica.cc:331] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748: stopping tablet replica
I20250904 13:59:17.493311 10634 raft_consensus.cc:2241] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:17.493530 14952 consensus_queue.cc:1035] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eae0557778094d6c9e874fb9f99ead31" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 44951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:59:17.494370 10634 raft_consensus.cc:2270] T 98d004e260ad46168f1321f718eef6a1 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.497100 10634 tablet_replica.cc:331] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748: stopping tablet replica
I20250904 13:59:17.497900 10634 raft_consensus.cc:2241] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:17.498833 10634 pending_rounds.cc:70] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748: Trying to abort 1 pending ops.
I20250904 13:59:17.499094 10634 pending_rounds.cc:77] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7196648886977515520 op_type: NO_OP noop_request { }
I20250904 13:59:17.499508 10634 raft_consensus.cc:2887] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20250904 13:59:17.499873 10634 raft_consensus.cc:2270] T f1fb57f4c29f42f3901517c38d4fd2bd P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.502835 10634 tablet_replica.cc:331] T 2746477d17a44175a64aed903c4cc706 P 1f0cdd7953ea4648a402cfaaa28ce748: stopping tablet replica
I20250904 13:59:17.503490 10634 raft_consensus.cc:2241] T 2746477d17a44175a64aed903c4cc706 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:17.504019 10634 raft_consensus.cc:2270] T 2746477d17a44175a64aed903c4cc706 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.506462 10634 tablet_replica.cc:331] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748: stopping tablet replica
I20250904 13:59:17.507002 10634 raft_consensus.cc:2241] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:17.507879 10634 raft_consensus.cc:2270] T 38244a7426a443efbd07ab8975ae0b75 P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.510227 10634 tablet_replica.cc:331] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748: stopping tablet replica
I20250904 13:59:17.510699 10634 raft_consensus.cc:2241] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:17.511431 10634 raft_consensus.cc:2270] T a93701c10d644072a8889318fd08c99a P 1f0cdd7953ea4648a402cfaaa28ce748 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.540035 10634 tablet_server.cc:195] TabletServer@127.10.98.129:0 shutdown complete.
I20250904 13:59:17.557870 10634 tablet_server.cc:178] TabletServer@127.10.98.130:0 shutting down...
W20250904 13:59:17.582810 14728 consensus_peers.cc:489] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b -> Peer 1f0cdd7953ea4648a402cfaaa28ce748 (127.10.98.129:43873): Couldn't send request to peer 1f0cdd7953ea4648a402cfaaa28ce748. Status: Network error: Client connection negotiation failed: client connection to 127.10.98.129:43873: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 13:59:17.585203 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:59:17.585894 10634 tablet_replica.cc:331] T 38244a7426a443efbd07ab8975ae0b75 P 6b21299357f942dab7c574db37593c3b: stopping tablet replica
I20250904 13:59:17.586587 10634 raft_consensus.cc:2241] T 38244a7426a443efbd07ab8975ae0b75 P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:17.587031 10634 raft_consensus.cc:2270] T 38244a7426a443efbd07ab8975ae0b75 P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.589298 10634 tablet_replica.cc:331] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b: stopping tablet replica
I20250904 13:59:17.589922 10634 raft_consensus.cc:2241] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:17.590880 10634 raft_consensus.cc:2270] T 2746477d17a44175a64aed903c4cc706 P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.593024 10634 tablet_replica.cc:331] T f1fb57f4c29f42f3901517c38d4fd2bd P 6b21299357f942dab7c574db37593c3b: stopping tablet replica
I20250904 13:59:17.593600 10634 raft_consensus.cc:2241] T f1fb57f4c29f42f3901517c38d4fd2bd P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:17.593990 10634 raft_consensus.cc:2270] T f1fb57f4c29f42f3901517c38d4fd2bd P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.595788 10634 tablet_replica.cc:331] T 4903d9835fcc4652b77ae055c23b8a4d P 6b21299357f942dab7c574db37593c3b: stopping tablet replica
I20250904 13:59:17.596338 10634 raft_consensus.cc:2241] T 4903d9835fcc4652b77ae055c23b8a4d P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:17.596762 10634 raft_consensus.cc:2270] T 4903d9835fcc4652b77ae055c23b8a4d P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.599005 10634 tablet_replica.cc:331] T 98d004e260ad46168f1321f718eef6a1 P 6b21299357f942dab7c574db37593c3b: stopping tablet replica
I20250904 13:59:17.599598 10634 raft_consensus.cc:2241] T 98d004e260ad46168f1321f718eef6a1 P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:17.600018 10634 raft_consensus.cc:2270] T 98d004e260ad46168f1321f718eef6a1 P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.602432 10634 tablet_replica.cc:331] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b: stopping tablet replica
I20250904 13:59:17.603025 10634 raft_consensus.cc:2241] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:17.603852 10634 raft_consensus.cc:2270] T 4eb02492ca4c4f0f990cb55611347087 P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.606179 10634 tablet_replica.cc:331] T a93701c10d644072a8889318fd08c99a P 6b21299357f942dab7c574db37593c3b: stopping tablet replica
I20250904 13:59:17.607035 10634 raft_consensus.cc:2241] T a93701c10d644072a8889318fd08c99a P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:17.607441 10634 raft_consensus.cc:2270] T a93701c10d644072a8889318fd08c99a P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.609534 10634 tablet_replica.cc:331] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b: stopping tablet replica
I20250904 13:59:17.610049 10634 raft_consensus.cc:2241] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:17.610900 10634 raft_consensus.cc:2270] T 68be2a2fe36345b4aaafe10bce4ab6cb P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.613219 10634 tablet_replica.cc:331] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b: stopping tablet replica
I20250904 13:59:17.614166 10634 raft_consensus.cc:2241] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:17.614965 10634 raft_consensus.cc:2270] T c69c4a3a981847d8a5febe6a619cc8a1 P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Raft consensus is shut down!
W20250904 13:59:17.615542 14801 consensus_peers.cc:489] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 -> Peer 6b21299357f942dab7c574db37593c3b (127.10.98.130:42551): Couldn't send request to peer 6b21299357f942dab7c574db37593c3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20250904 13:59:17.617151 10634 tablet_replica.cc:331] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b: stopping tablet replica
I20250904 13:59:17.617636 10634 raft_consensus.cc:2241] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:17.618453 10634 raft_consensus.cc:2270] T 77ea8776d09e48978e66a1c36c15cb56 P 6b21299357f942dab7c574db37593c3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.648172 10634 tablet_server.cc:195] TabletServer@127.10.98.130:0 shutdown complete.
I20250904 13:59:17.665721 10634 tablet_server.cc:178] TabletServer@127.10.98.131:0 shutting down...
W20250904 13:59:17.692051 14802 consensus_peers.cc:489] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 -> Peer 1f0cdd7953ea4648a402cfaaa28ce748 (127.10.98.129:43873): Couldn't send request to peer 1f0cdd7953ea4648a402cfaaa28ce748. Status: Network error: Client connection negotiation failed: client connection to 127.10.98.129:43873: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250904 13:59:17.693842 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:59:17.694604 10634 tablet_replica.cc:331] T f1fb57f4c29f42f3901517c38d4fd2bd P eae0557778094d6c9e874fb9f99ead31: stopping tablet replica
I20250904 13:59:17.695254 10634 raft_consensus.cc:2241] T f1fb57f4c29f42f3901517c38d4fd2bd P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:17.695662 10634 raft_consensus.cc:2270] T f1fb57f4c29f42f3901517c38d4fd2bd P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.697467 10634 tablet_replica.cc:331] T 2746477d17a44175a64aed903c4cc706 P eae0557778094d6c9e874fb9f99ead31: stopping tablet replica
I20250904 13:59:17.698047 10634 raft_consensus.cc:2241] T 2746477d17a44175a64aed903c4cc706 P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:17.698561 10634 raft_consensus.cc:2270] T 2746477d17a44175a64aed903c4cc706 P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.701025 10634 tablet_replica.cc:331] T 98d004e260ad46168f1321f718eef6a1 P eae0557778094d6c9e874fb9f99ead31: stopping tablet replica
I20250904 13:59:17.701606 10634 raft_consensus.cc:2241] T 98d004e260ad46168f1321f718eef6a1 P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:17.702026 10634 raft_consensus.cc:2270] T 98d004e260ad46168f1321f718eef6a1 P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.704397 10634 tablet_replica.cc:331] T 77ea8776d09e48978e66a1c36c15cb56 P eae0557778094d6c9e874fb9f99ead31: stopping tablet replica
I20250904 13:59:17.704999 10634 raft_consensus.cc:2241] T 77ea8776d09e48978e66a1c36c15cb56 P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:17.705410 10634 raft_consensus.cc:2270] T 77ea8776d09e48978e66a1c36c15cb56 P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.707641 10634 tablet_replica.cc:331] T 68be2a2fe36345b4aaafe10bce4ab6cb P eae0557778094d6c9e874fb9f99ead31: stopping tablet replica
I20250904 13:59:17.708200 10634 raft_consensus.cc:2241] T 68be2a2fe36345b4aaafe10bce4ab6cb P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:17.708611 10634 raft_consensus.cc:2270] T 68be2a2fe36345b4aaafe10bce4ab6cb P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.710914 10634 tablet_replica.cc:331] T c69c4a3a981847d8a5febe6a619cc8a1 P eae0557778094d6c9e874fb9f99ead31: stopping tablet replica
I20250904 13:59:17.711460 10634 raft_consensus.cc:2241] T c69c4a3a981847d8a5febe6a619cc8a1 P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:17.711877 10634 raft_consensus.cc:2270] T c69c4a3a981847d8a5febe6a619cc8a1 P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.714192 10634 tablet_replica.cc:331] T 38244a7426a443efbd07ab8975ae0b75 P eae0557778094d6c9e874fb9f99ead31: stopping tablet replica
I20250904 13:59:17.714740 10634 raft_consensus.cc:2241] T 38244a7426a443efbd07ab8975ae0b75 P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:17.715139 10634 raft_consensus.cc:2270] T 38244a7426a443efbd07ab8975ae0b75 P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.717242 10634 tablet_replica.cc:331] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31: stopping tablet replica
I20250904 13:59:17.717747 10634 raft_consensus.cc:2241] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:17.718616 10634 raft_consensus.cc:2270] T 4903d9835fcc4652b77ae055c23b8a4d P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.720582 10634 tablet_replica.cc:331] T a93701c10d644072a8889318fd08c99a P eae0557778094d6c9e874fb9f99ead31: stopping tablet replica
I20250904 13:59:17.721091 10634 raft_consensus.cc:2241] T a93701c10d644072a8889318fd08c99a P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:17.721482 10634 raft_consensus.cc:2270] T a93701c10d644072a8889318fd08c99a P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.723639 10634 tablet_replica.cc:331] T 4eb02492ca4c4f0f990cb55611347087 P eae0557778094d6c9e874fb9f99ead31: stopping tablet replica
I20250904 13:59:17.724112 10634 raft_consensus.cc:2241] T 4eb02492ca4c4f0f990cb55611347087 P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:17.724488 10634 raft_consensus.cc:2270] T 4eb02492ca4c4f0f990cb55611347087 P eae0557778094d6c9e874fb9f99ead31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.744304 10634 tablet_server.cc:195] TabletServer@127.10.98.131:0 shutdown complete.
I20250904 13:59:17.762085 10634 master.cc:561] Master@127.10.98.190:34049 shutting down...
I20250904 13:59:17.779920 10634 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:17.780459 10634 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:17.780709 10634 tablet_replica.cc:331] T 00000000000000000000000000000000 P 3bd50f780abc4ef497d8a29d5d5feac2: stopping tablet replica
I20250904 13:59:17.801101 10634 master.cc:583] Master@127.10.98.190:34049 shutdown complete.
[ OK ] NumReplicas/TServerQuiescingParamITest.TestNoElectionsForNewReplicas/1 (3807 ms)
[ RUN ] NumReplicas/TServerQuiescingParamITest.TestScansRetry/0
I20250904 13:59:17.836025 10634 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.98.190:44933
I20250904 13:59:17.837127 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:17.842381 14977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:17.843348 14976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:17.844374 10634 server_base.cc:1047] running on GCE node
W20250904 13:59:17.844583 14979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:17.845511 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:17.845731 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:17.845880 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994357845863 us; error 0 us; skew 500 ppm
I20250904 13:59:17.846414 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:17.848801 10634 webserver.cc:480] Webserver started at http://127.10.98.190:37777/ using document root <none> and password file <none>
I20250904 13:59:17.849303 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:17.849486 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:17.849747 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:17.851117 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_0.1756994241738102-10634-0/minicluster-data/master-0-root/instance:
uuid: "01e02e3cb0674dbdb480d8f5ebdaf125"
format_stamp: "Formatted at 2025-09-04 13:59:17 on dist-test-slave-jx76"
I20250904 13:59:17.855898 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250904 13:59:17.859488 14984 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:17.860268 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250904 13:59:17.860584 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_0.1756994241738102-10634-0/minicluster-data/master-0-root
uuid: "01e02e3cb0674dbdb480d8f5ebdaf125"
format_stamp: "Formatted at 2025-09-04 13:59:17 on dist-test-slave-jx76"
I20250904 13:59:17.860874 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_0.1756994241738102-10634-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_0.1756994241738102-10634-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_0.1756994241738102-10634-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:17.875236 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:17.876325 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:17.915668 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.190:44933
I20250904 13:59:17.915773 15035 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.190:44933 every 8 connection(s)
I20250904 13:59:17.919692 15036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:17.931695 15036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125: Bootstrap starting.
I20250904 13:59:17.936296 15036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:17.940541 15036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125: No bootstrap required, opened a new log
I20250904 13:59:17.942679 15036 raft_consensus.cc:357] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e02e3cb0674dbdb480d8f5ebdaf125" member_type: VOTER }
I20250904 13:59:17.943120 15036 raft_consensus.cc:383] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:17.943359 15036 raft_consensus.cc:738] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01e02e3cb0674dbdb480d8f5ebdaf125, State: Initialized, Role: FOLLOWER
I20250904 13:59:17.943994 15036 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e02e3cb0674dbdb480d8f5ebdaf125" member_type: VOTER }
I20250904 13:59:17.944468 15036 raft_consensus.cc:397] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:59:17.944687 15036 raft_consensus.cc:491] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250904 13:59:17.944945 15036 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:17.950356 15036 raft_consensus.cc:513] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e02e3cb0674dbdb480d8f5ebdaf125" member_type: VOTER }
I20250904 13:59:17.950961 15036 leader_election.cc:304] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01e02e3cb0674dbdb480d8f5ebdaf125; no voters:
I20250904 13:59:17.952193 15036 leader_election.cc:290] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:59:17.952564 15039 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:17.953922 15039 raft_consensus.cc:695] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125 [term 1 LEADER]: Becoming Leader. State: Replica: 01e02e3cb0674dbdb480d8f5ebdaf125, State: Running, Role: LEADER
I20250904 13:59:17.954668 15039 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e02e3cb0674dbdb480d8f5ebdaf125" member_type: VOTER }
I20250904 13:59:17.955371 15036 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 13:59:17.958870 15040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01e02e3cb0674dbdb480d8f5ebdaf125" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e02e3cb0674dbdb480d8f5ebdaf125" member_type: VOTER } }
I20250904 13:59:17.958777 15041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01e02e3cb0674dbdb480d8f5ebdaf125. Latest consensus state: current_term: 1 leader_uuid: "01e02e3cb0674dbdb480d8f5ebdaf125" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e02e3cb0674dbdb480d8f5ebdaf125" member_type: VOTER } }
I20250904 13:59:17.959482 15040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125 [sys.catalog]: This master's current role is: LEADER
I20250904 13:59:17.959496 15041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125 [sys.catalog]: This master's current role is: LEADER
I20250904 13:59:17.967216 15045 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 13:59:17.973243 15045 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 13:59:17.974529 10634 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 13:59:17.982280 15045 catalog_manager.cc:1349] Generated new cluster ID: b91b74dee8a14cda95abff2e32620106
I20250904 13:59:17.982652 15045 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 13:59:18.006351 15045 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 13:59:18.007858 15045 catalog_manager.cc:1506] Loading token signing keys...
I20250904 13:59:18.024042 15045 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125: Generated new TSK 0
I20250904 13:59:18.024771 15045 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 13:59:18.041482 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:18.047776 15057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:18.048830 15058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:18.052902 10634 server_base.cc:1047] running on GCE node
W20250904 13:59:18.052860 15060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:18.054330 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:18.054550 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:18.054687 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994358054675 us; error 0 us; skew 500 ppm
I20250904 13:59:18.055179 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:18.057754 10634 webserver.cc:480] Webserver started at http://127.10.98.129:42887/ using document root <none> and password file <none>
I20250904 13:59:18.058334 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:18.058532 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:18.058764 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:18.059967 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_0.1756994241738102-10634-0/minicluster-data/ts-0-root/instance:
uuid: "4e503cdc37d04b0989242a02320982f3"
format_stamp: "Formatted at 2025-09-04 13:59:18 on dist-test-slave-jx76"
I20250904 13:59:18.065177 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20250904 13:59:18.069218 15065 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:18.070050 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250904 13:59:18.070408 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_0.1756994241738102-10634-0/minicluster-data/ts-0-root
uuid: "4e503cdc37d04b0989242a02320982f3"
format_stamp: "Formatted at 2025-09-04 13:59:18 on dist-test-slave-jx76"
I20250904 13:59:18.070746 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_0.1756994241738102-10634-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_0.1756994241738102-10634-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_0.1756994241738102-10634-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:18.097254 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:18.098527 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:18.100052 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:59:18.102624 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:59:18.102842 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:18.103093 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:59:18.103247 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:18.143987 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.129:35241
I20250904 13:59:18.144083 15127 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.129:35241 every 8 connection(s)
I20250904 13:59:18.159006 15128 heartbeater.cc:344] Connected to a master server at 127.10.98.190:44933
I20250904 13:59:18.159394 15128 heartbeater.cc:461] Registering TS with master...
I20250904 13:59:18.160157 15128 heartbeater.cc:507] Master 127.10.98.190:44933 requested a full tablet report, sending...
I20250904 13:59:18.162222 15001 ts_manager.cc:194] Registered new tserver with Master: 4e503cdc37d04b0989242a02320982f3 (127.10.98.129:35241)
I20250904 13:59:18.163010 10634 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.015530576s
I20250904 13:59:18.164183 15001 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44644
I20250904 13:59:18.196298 10634 test_util.cc:276] Using random seed: 546814001
I20250904 13:59:18.220597 15001 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44674:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250904 13:59:18.253074 15093 tablet_service.cc:1468] Processing CreateTablet for tablet febc1481d7384d728346579b708de079 (DEFAULT_TABLE table=test-workload [id=82561c5c6fe34e1d90e36092a1310913]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:59:18.254478 15093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet febc1481d7384d728346579b708de079. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:18.269826 15146 tablet_bootstrap.cc:492] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3: Bootstrap starting.
I20250904 13:59:18.274829 15146 tablet_bootstrap.cc:654] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:18.279364 15146 tablet_bootstrap.cc:492] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3: No bootstrap required, opened a new log
I20250904 13:59:18.279807 15146 ts_tablet_manager.cc:1397] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20250904 13:59:18.281728 15146 raft_consensus.cc:357] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e503cdc37d04b0989242a02320982f3" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35241 } }
I20250904 13:59:18.282279 15146 raft_consensus.cc:383] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:18.282568 15146 raft_consensus.cc:738] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e503cdc37d04b0989242a02320982f3, State: Initialized, Role: FOLLOWER
I20250904 13:59:18.283216 15146 consensus_queue.cc:260] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e503cdc37d04b0989242a02320982f3" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35241 } }
I20250904 13:59:18.283751 15146 raft_consensus.cc:397] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:59:18.283987 15146 raft_consensus.cc:491] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250904 13:59:18.284303 15146 raft_consensus.cc:3058] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:18.290153 15146 raft_consensus.cc:513] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e503cdc37d04b0989242a02320982f3" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35241 } }
I20250904 13:59:18.290791 15146 leader_election.cc:304] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e503cdc37d04b0989242a02320982f3; no voters:
I20250904 13:59:18.292075 15146 leader_election.cc:290] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:59:18.292531 15148 raft_consensus.cc:2802] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:18.294538 15128 heartbeater.cc:499] Master 127.10.98.190:44933 was elected leader, sending a full tablet report...
I20250904 13:59:18.295040 15146 ts_tablet_manager.cc:1428] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20250904 13:59:18.295737 15148 raft_consensus.cc:695] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3 [term 1 LEADER]: Becoming Leader. State: Replica: 4e503cdc37d04b0989242a02320982f3, State: Running, Role: LEADER
I20250904 13:59:18.296451 15148 consensus_queue.cc:237] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e503cdc37d04b0989242a02320982f3" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35241 } }
I20250904 13:59:18.303820 15001 catalog_manager.cc:5582] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e503cdc37d04b0989242a02320982f3 (127.10.98.129). New cstate: current_term: 1 leader_uuid: "4e503cdc37d04b0989242a02320982f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e503cdc37d04b0989242a02320982f3" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 35241 } health_report { overall_health: HEALTHY } } }
I20250904 13:59:23.264015 10634 scanner-internal.cc:172] Scan of tablet febc1481d7384d728346579b708de079 at 4e503cdc37d04b0989242a02320982f3 (127.10.98.129:35241) deadline exceeded after 1.000s
W20250904 13:59:23.265033 10634 scanner-internal.cc:458] Time spent opening tablet: real 1.225s user 0.007s sys 0.001s
I20250904 13:59:25.364172 10634 tablet_server.cc:178] TabletServer@127.10.98.129:0 shutting down...
I20250904 13:59:25.407042 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:59:25.407743 10634 tablet_replica.cc:331] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3: stopping tablet replica
I20250904 13:59:25.408452 10634 raft_consensus.cc:2241] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:25.409350 10634 raft_consensus.cc:2270] T febc1481d7384d728346579b708de079 P 4e503cdc37d04b0989242a02320982f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:25.455276 10634 tablet_server.cc:195] TabletServer@127.10.98.129:0 shutdown complete.
I20250904 13:59:25.475250 10634 master.cc:561] Master@127.10.98.190:44933 shutting down...
I20250904 13:59:25.499155 10634 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:25.499991 10634 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:25.500362 10634 tablet_replica.cc:331] T 00000000000000000000000000000000 P 01e02e3cb0674dbdb480d8f5ebdaf125: stopping tablet replica
W20250904 13:59:26.161353 15032 debug-util.cc:398] Leaking SignalData structure 0x7b0800029660 after lost signal to thread 10637
W20250904 13:59:26.518988 10634 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 15032)
W20250904 13:59:27.519682 10634 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 15032)
I20250904 13:59:28.373345 10634 master.cc:583] Master@127.10.98.190:44933 shutdown complete.
[ OK ] NumReplicas/TServerQuiescingParamITest.TestScansRetry/0 (10579 ms)
[ RUN ] NumReplicas/TServerQuiescingParamITest.TestScansRetry/1
I20250904 13:59:28.415596 10634 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.98.190:39647
I20250904 13:59:28.416815 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:28.422147 15169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:28.423848 15172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:28.424439 10634 server_base.cc:1047] running on GCE node
W20250904 13:59:28.424392 15170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:28.425534 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:28.425750 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:28.425902 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994368425881 us; error 0 us; skew 500 ppm
I20250904 13:59:28.426431 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:28.428903 10634 webserver.cc:480] Webserver started at http://127.10.98.190:35909/ using document root <none> and password file <none>
I20250904 13:59:28.429412 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:28.429601 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:28.429970 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:28.431310 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_1.1756994241738102-10634-0/minicluster-data/master-0-root/instance:
uuid: "2f31409816d64acb8621783fdbe35fd2"
format_stamp: "Formatted at 2025-09-04 13:59:28 on dist-test-slave-jx76"
I20250904 13:59:28.436220 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250904 13:59:28.439978 15177 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:28.440835 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20250904 13:59:28.441092 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_1.1756994241738102-10634-0/minicluster-data/master-0-root
uuid: "2f31409816d64acb8621783fdbe35fd2"
format_stamp: "Formatted at 2025-09-04 13:59:28 on dist-test-slave-jx76"
I20250904 13:59:28.441339 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_1.1756994241738102-10634-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_1.1756994241738102-10634-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_1.1756994241738102-10634-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:28.453078 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:28.454442 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:28.496368 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.190:39647
I20250904 13:59:28.496500 15228 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.190:39647 every 8 connection(s)
I20250904 13:59:28.500830 15229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:28.514249 15229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2: Bootstrap starting.
I20250904 13:59:28.519305 15229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:28.523994 15229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2: No bootstrap required, opened a new log
I20250904 13:59:28.526381 15229 raft_consensus.cc:357] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f31409816d64acb8621783fdbe35fd2" member_type: VOTER }
I20250904 13:59:28.526845 15229 raft_consensus.cc:383] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:28.527091 15229 raft_consensus.cc:738] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f31409816d64acb8621783fdbe35fd2, State: Initialized, Role: FOLLOWER
I20250904 13:59:28.527738 15229 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f31409816d64acb8621783fdbe35fd2" member_type: VOTER }
I20250904 13:59:28.528204 15229 raft_consensus.cc:397] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:59:28.528466 15229 raft_consensus.cc:491] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250904 13:59:28.528748 15229 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:28.534222 15229 raft_consensus.cc:513] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f31409816d64acb8621783fdbe35fd2" member_type: VOTER }
I20250904 13:59:28.534821 15229 leader_election.cc:304] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f31409816d64acb8621783fdbe35fd2; no voters:
I20250904 13:59:28.536095 15229 leader_election.cc:290] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:59:28.536556 15232 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:28.537938 15232 raft_consensus.cc:695] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2 [term 1 LEADER]: Becoming Leader. State: Replica: 2f31409816d64acb8621783fdbe35fd2, State: Running, Role: LEADER
I20250904 13:59:28.538687 15232 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f31409816d64acb8621783fdbe35fd2" member_type: VOTER }
I20250904 13:59:28.539273 15229 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 13:59:28.543831 15234 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f31409816d64acb8621783fdbe35fd2. Latest consensus state: current_term: 1 leader_uuid: "2f31409816d64acb8621783fdbe35fd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f31409816d64acb8621783fdbe35fd2" member_type: VOTER } }
I20250904 13:59:28.543872 15233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f31409816d64acb8621783fdbe35fd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f31409816d64acb8621783fdbe35fd2" member_type: VOTER } }
I20250904 13:59:28.544935 15234 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2 [sys.catalog]: This master's current role is: LEADER
I20250904 13:59:28.545128 15233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2 [sys.catalog]: This master's current role is: LEADER
I20250904 13:59:28.551227 15238 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 13:59:28.556413 15238 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 13:59:28.558430 10634 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 13:59:28.566293 15238 catalog_manager.cc:1349] Generated new cluster ID: 9c11d53215234798ae99bd26505b6e3a
I20250904 13:59:28.566583 15238 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 13:59:28.589943 15238 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 13:59:28.591480 15238 catalog_manager.cc:1506] Loading token signing keys...
I20250904 13:59:28.616473 15238 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2: Generated new TSK 0
I20250904 13:59:28.617131 15238 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 13:59:28.625283 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:28.631688 15250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:28.633050 15251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:28.635040 15253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:28.635933 10634 server_base.cc:1047] running on GCE node
I20250904 13:59:28.636848 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:28.637045 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:28.637194 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994368637178 us; error 0 us; skew 500 ppm
I20250904 13:59:28.637686 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:28.640264 10634 webserver.cc:480] Webserver started at http://127.10.98.129:43377/ using document root <none> and password file <none>
I20250904 13:59:28.640820 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:28.640998 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:28.641252 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:28.642653 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_1.1756994241738102-10634-0/minicluster-data/ts-0-root/instance:
uuid: "f36679231d5d4b89bb3d043493457566"
format_stamp: "Formatted at 2025-09-04 13:59:28 on dist-test-slave-jx76"
I20250904 13:59:28.647665 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20250904 13:59:28.651294 15258 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:28.652196 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20250904 13:59:28.652515 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_1.1756994241738102-10634-0/minicluster-data/ts-0-root
uuid: "f36679231d5d4b89bb3d043493457566"
format_stamp: "Formatted at 2025-09-04 13:59:28 on dist-test-slave-jx76"
I20250904 13:59:28.652793 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_1.1756994241738102-10634-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_1.1756994241738102-10634-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_1.1756994241738102-10634-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:28.685839 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:28.687127 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:28.688637 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:59:28.691105 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:59:28.691296 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:28.691593 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:59:28.691762 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:28.735847 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.129:44521
I20250904 13:59:28.735954 15320 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.129:44521 every 8 connection(s)
I20250904 13:59:28.741043 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:28.749096 15325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:28.749802 15326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:28.758684 15328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:28.759608 10634 server_base.cc:1047] running on GCE node
I20250904 13:59:28.761225 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:28.761452 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:28.761621 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994368761600 us; error 0 us; skew 500 ppm
I20250904 13:59:28.762208 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:28.765859 10634 webserver.cc:480] Webserver started at http://127.10.98.130:34455/ using document root <none> and password file <none>
I20250904 13:59:28.766011 15321 heartbeater.cc:344] Connected to a master server at 127.10.98.190:39647
I20250904 13:59:28.766516 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:28.766513 15321 heartbeater.cc:461] Registering TS with master...
I20250904 13:59:28.766839 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:28.767143 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:28.767536 15321 heartbeater.cc:507] Master 127.10.98.190:39647 requested a full tablet report, sending...
I20250904 13:59:28.768534 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_1.1756994241738102-10634-0/minicluster-data/ts-1-root/instance:
uuid: "b6c8f6ed18564f06b157bc0be92184bf"
format_stamp: "Formatted at 2025-09-04 13:59:28 on dist-test-slave-jx76"
I20250904 13:59:28.769771 15194 ts_manager.cc:194] Registered new tserver with Master: f36679231d5d4b89bb3d043493457566 (127.10.98.129:44521)
I20250904 13:59:28.771847 15194 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48132
I20250904 13:59:28.773944 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20250904 13:59:28.778107 15333 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:28.778973 10634 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20250904 13:59:28.779282 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_1.1756994241738102-10634-0/minicluster-data/ts-1-root
uuid: "b6c8f6ed18564f06b157bc0be92184bf"
format_stamp: "Formatted at 2025-09-04 13:59:28 on dist-test-slave-jx76"
I20250904 13:59:28.779577 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_1.1756994241738102-10634-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_1.1756994241738102-10634-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_1.1756994241738102-10634-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:28.790891 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:28.792197 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:28.793788 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:59:28.796247 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:59:28.796485 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:28.796732 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:59:28.796896 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:28.842299 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.130:41707
I20250904 13:59:28.842432 15395 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.130:41707 every 8 connection(s)
I20250904 13:59:28.847811 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:28.855993 15399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:28.857982 15400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:28.860006 10634 server_base.cc:1047] running on GCE node
W20250904 13:59:28.859983 15402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:28.861310 15396 heartbeater.cc:344] Connected to a master server at 127.10.98.190:39647
I20250904 13:59:28.861688 15396 heartbeater.cc:461] Registering TS with master...
I20250904 13:59:28.861750 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:28.862035 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:28.862244 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994368862227 us; error 0 us; skew 500 ppm
I20250904 13:59:28.862582 15396 heartbeater.cc:507] Master 127.10.98.190:39647 requested a full tablet report, sending...
I20250904 13:59:28.862875 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:28.864648 15194 ts_manager.cc:194] Registered new tserver with Master: b6c8f6ed18564f06b157bc0be92184bf (127.10.98.130:41707)
I20250904 13:59:28.865926 10634 webserver.cc:480] Webserver started at http://127.10.98.131:35373/ using document root <none> and password file <none>
I20250904 13:59:28.866302 15194 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48138
I20250904 13:59:28.866532 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:28.866802 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:28.867070 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:28.868526 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_1.1756994241738102-10634-0/minicluster-data/ts-2-root/instance:
uuid: "40a9ee18d4024a57b8068cf872d335d2"
format_stamp: "Formatted at 2025-09-04 13:59:28 on dist-test-slave-jx76"
I20250904 13:59:28.873317 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20250904 13:59:28.877079 15407 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:28.877888 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250904 13:59:28.878237 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_1.1756994241738102-10634-0/minicluster-data/ts-2-root
uuid: "40a9ee18d4024a57b8068cf872d335d2"
format_stamp: "Formatted at 2025-09-04 13:59:28 on dist-test-slave-jx76"
I20250904 13:59:28.878513 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_1.1756994241738102-10634-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_1.1756994241738102-10634-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestScansRetry_1.1756994241738102-10634-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:28.892941 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:28.894290 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:28.895870 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:59:28.898391 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:59:28.898598 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:28.898882 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:59:28.899030 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:28.943300 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.131:35351
I20250904 13:59:28.943434 15469 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.131:35351 every 8 connection(s)
I20250904 13:59:28.958842 15470 heartbeater.cc:344] Connected to a master server at 127.10.98.190:39647
I20250904 13:59:28.959273 15470 heartbeater.cc:461] Registering TS with master...
I20250904 13:59:28.960107 15470 heartbeater.cc:507] Master 127.10.98.190:39647 requested a full tablet report, sending...
I20250904 13:59:28.962181 15194 ts_manager.cc:194] Registered new tserver with Master: 40a9ee18d4024a57b8068cf872d335d2 (127.10.98.131:35351)
I20250904 13:59:28.963066 10634 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016147481s
I20250904 13:59:28.964040 15194 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48152
I20250904 13:59:29.000025 10634 test_util.cc:276] Using random seed: 557617727
I20250904 13:59:29.023950 15194 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48188:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 13:59:29.026104 15194 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 13:59:29.066596 15361 tablet_service.cc:1468] Processing CreateTablet for tablet 5c3377ee8321447dacf77ca638c291e9 (DEFAULT_TABLE table=test-workload [id=8df228155e644df9beb43724dc82e0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:59:29.068053 15361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c3377ee8321447dacf77ca638c291e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:29.069499 15286 tablet_service.cc:1468] Processing CreateTablet for tablet 5c3377ee8321447dacf77ca638c291e9 (DEFAULT_TABLE table=test-workload [id=8df228155e644df9beb43724dc82e0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:59:29.070925 15286 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c3377ee8321447dacf77ca638c291e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:29.074777 15435 tablet_service.cc:1468] Processing CreateTablet for tablet 5c3377ee8321447dacf77ca638c291e9 (DEFAULT_TABLE table=test-workload [id=8df228155e644df9beb43724dc82e0da]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:59:29.076269 15435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5c3377ee8321447dacf77ca638c291e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:29.091310 15492 tablet_bootstrap.cc:492] T 5c3377ee8321447dacf77ca638c291e9 P f36679231d5d4b89bb3d043493457566: Bootstrap starting.
I20250904 13:59:29.093928 15491 tablet_bootstrap.cc:492] T 5c3377ee8321447dacf77ca638c291e9 P b6c8f6ed18564f06b157bc0be92184bf: Bootstrap starting.
I20250904 13:59:29.098970 15493 tablet_bootstrap.cc:492] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2: Bootstrap starting.
I20250904 13:59:29.100481 15492 tablet_bootstrap.cc:654] T 5c3377ee8321447dacf77ca638c291e9 P f36679231d5d4b89bb3d043493457566: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:29.102207 15491 tablet_bootstrap.cc:654] T 5c3377ee8321447dacf77ca638c291e9 P b6c8f6ed18564f06b157bc0be92184bf: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:29.105764 15492 tablet_bootstrap.cc:492] T 5c3377ee8321447dacf77ca638c291e9 P f36679231d5d4b89bb3d043493457566: No bootstrap required, opened a new log
I20250904 13:59:29.106407 15492 ts_tablet_manager.cc:1397] T 5c3377ee8321447dacf77ca638c291e9 P f36679231d5d4b89bb3d043493457566: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.003s
I20250904 13:59:29.107088 15493 tablet_bootstrap.cc:654] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:29.109849 15491 tablet_bootstrap.cc:492] T 5c3377ee8321447dacf77ca638c291e9 P b6c8f6ed18564f06b157bc0be92184bf: No bootstrap required, opened a new log
I20250904 13:59:29.109408 15492 raft_consensus.cc:357] T 5c3377ee8321447dacf77ca638c291e9 P f36679231d5d4b89bb3d043493457566 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c8f6ed18564f06b157bc0be92184bf" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41707 } } peers { permanent_uuid: "f36679231d5d4b89bb3d043493457566" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 44521 } } peers { permanent_uuid: "40a9ee18d4024a57b8068cf872d335d2" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 35351 } }
I20250904 13:59:29.110422 15492 raft_consensus.cc:383] T 5c3377ee8321447dacf77ca638c291e9 P f36679231d5d4b89bb3d043493457566 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:29.110467 15491 ts_tablet_manager.cc:1397] T 5c3377ee8321447dacf77ca638c291e9 P b6c8f6ed18564f06b157bc0be92184bf: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.009s
I20250904 13:59:29.110846 15492 raft_consensus.cc:738] T 5c3377ee8321447dacf77ca638c291e9 P f36679231d5d4b89bb3d043493457566 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f36679231d5d4b89bb3d043493457566, State: Initialized, Role: FOLLOWER
I20250904 13:59:29.111778 15492 consensus_queue.cc:260] T 5c3377ee8321447dacf77ca638c291e9 P f36679231d5d4b89bb3d043493457566 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c8f6ed18564f06b157bc0be92184bf" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41707 } } peers { permanent_uuid: "f36679231d5d4b89bb3d043493457566" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 44521 } } peers { permanent_uuid: "40a9ee18d4024a57b8068cf872d335d2" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 35351 } }
I20250904 13:59:29.113658 15493 tablet_bootstrap.cc:492] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2: No bootstrap required, opened a new log
I20250904 13:59:29.114241 15493 ts_tablet_manager.cc:1397] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.006s
I20250904 13:59:29.114670 15321 heartbeater.cc:499] Master 127.10.98.190:39647 was elected leader, sending a full tablet report...
I20250904 13:59:29.114836 15491 raft_consensus.cc:357] T 5c3377ee8321447dacf77ca638c291e9 P b6c8f6ed18564f06b157bc0be92184bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c8f6ed18564f06b157bc0be92184bf" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41707 } } peers { permanent_uuid: "f36679231d5d4b89bb3d043493457566" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 44521 } } peers { permanent_uuid: "40a9ee18d4024a57b8068cf872d335d2" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 35351 } }
I20250904 13:59:29.115439 15492 ts_tablet_manager.cc:1428] T 5c3377ee8321447dacf77ca638c291e9 P f36679231d5d4b89bb3d043493457566: Time spent starting tablet: real 0.009s user 0.006s sys 0.002s
I20250904 13:59:29.115690 15491 raft_consensus.cc:383] T 5c3377ee8321447dacf77ca638c291e9 P b6c8f6ed18564f06b157bc0be92184bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:29.116025 15491 raft_consensus.cc:738] T 5c3377ee8321447dacf77ca638c291e9 P b6c8f6ed18564f06b157bc0be92184bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6c8f6ed18564f06b157bc0be92184bf, State: Initialized, Role: FOLLOWER
I20250904 13:59:29.116793 15491 consensus_queue.cc:260] T 5c3377ee8321447dacf77ca638c291e9 P b6c8f6ed18564f06b157bc0be92184bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c8f6ed18564f06b157bc0be92184bf" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41707 } } peers { permanent_uuid: "f36679231d5d4b89bb3d043493457566" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 44521 } } peers { permanent_uuid: "40a9ee18d4024a57b8068cf872d335d2" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 35351 } }
I20250904 13:59:29.117329 15493 raft_consensus.cc:357] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c8f6ed18564f06b157bc0be92184bf" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41707 } } peers { permanent_uuid: "f36679231d5d4b89bb3d043493457566" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 44521 } } peers { permanent_uuid: "40a9ee18d4024a57b8068cf872d335d2" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 35351 } }
I20250904 13:59:29.118155 15493 raft_consensus.cc:383] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:29.118571 15493 raft_consensus.cc:738] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a9ee18d4024a57b8068cf872d335d2, State: Initialized, Role: FOLLOWER
I20250904 13:59:29.119517 15493 consensus_queue.cc:260] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c8f6ed18564f06b157bc0be92184bf" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41707 } } peers { permanent_uuid: "f36679231d5d4b89bb3d043493457566" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 44521 } } peers { permanent_uuid: "40a9ee18d4024a57b8068cf872d335d2" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 35351 } }
I20250904 13:59:29.120239 15491 ts_tablet_manager.cc:1428] T 5c3377ee8321447dacf77ca638c291e9 P b6c8f6ed18564f06b157bc0be92184bf: Time spent starting tablet: real 0.009s user 0.001s sys 0.006s
I20250904 13:59:29.121881 15396 heartbeater.cc:499] Master 127.10.98.190:39647 was elected leader, sending a full tablet report...
I20250904 13:59:29.122288 15470 heartbeater.cc:499] Master 127.10.98.190:39647 was elected leader, sending a full tablet report...
I20250904 13:59:29.122970 15493 ts_tablet_manager.cc:1428] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2: Time spent starting tablet: real 0.008s user 0.007s sys 0.001s
I20250904 13:59:29.495815 15499 raft_consensus.cc:491] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:29.496322 15499 raft_consensus.cc:513] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c8f6ed18564f06b157bc0be92184bf" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41707 } } peers { permanent_uuid: "f36679231d5d4b89bb3d043493457566" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 44521 } } peers { permanent_uuid: "40a9ee18d4024a57b8068cf872d335d2" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 35351 } }
I20250904 13:59:29.498660 15499 leader_election.cc:290] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6c8f6ed18564f06b157bc0be92184bf (127.10.98.130:41707), f36679231d5d4b89bb3d043493457566 (127.10.98.129:44521)
I20250904 13:59:29.510470 15371 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5c3377ee8321447dacf77ca638c291e9" candidate_uuid: "40a9ee18d4024a57b8068cf872d335d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6c8f6ed18564f06b157bc0be92184bf" is_pre_election: true
I20250904 13:59:29.510699 15296 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5c3377ee8321447dacf77ca638c291e9" candidate_uuid: "40a9ee18d4024a57b8068cf872d335d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f36679231d5d4b89bb3d043493457566" is_pre_election: true
I20250904 13:59:29.511322 15371 raft_consensus.cc:2466] T 5c3377ee8321447dacf77ca638c291e9 P b6c8f6ed18564f06b157bc0be92184bf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40a9ee18d4024a57b8068cf872d335d2 in term 0.
I20250904 13:59:29.511402 15296 raft_consensus.cc:2466] T 5c3377ee8321447dacf77ca638c291e9 P f36679231d5d4b89bb3d043493457566 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40a9ee18d4024a57b8068cf872d335d2 in term 0.
I20250904 13:59:29.512629 15409 leader_election.cc:304] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a9ee18d4024a57b8068cf872d335d2, b6c8f6ed18564f06b157bc0be92184bf; no voters:
I20250904 13:59:29.513437 15499 raft_consensus.cc:2802] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:29.513798 15499 raft_consensus.cc:491] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:29.514047 15499 raft_consensus.cc:3058] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:29.521067 15499 raft_consensus.cc:513] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c8f6ed18564f06b157bc0be92184bf" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41707 } } peers { permanent_uuid: "f36679231d5d4b89bb3d043493457566" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 44521 } } peers { permanent_uuid: "40a9ee18d4024a57b8068cf872d335d2" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 35351 } }
I20250904 13:59:29.522998 15499 leader_election.cc:290] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [CANDIDATE]: Term 1 election: Requested vote from peers b6c8f6ed18564f06b157bc0be92184bf (127.10.98.130:41707), f36679231d5d4b89bb3d043493457566 (127.10.98.129:44521)
I20250904 13:59:29.523959 15371 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5c3377ee8321447dacf77ca638c291e9" candidate_uuid: "40a9ee18d4024a57b8068cf872d335d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6c8f6ed18564f06b157bc0be92184bf"
I20250904 13:59:29.524127 15296 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "5c3377ee8321447dacf77ca638c291e9" candidate_uuid: "40a9ee18d4024a57b8068cf872d335d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f36679231d5d4b89bb3d043493457566"
I20250904 13:59:29.524524 15371 raft_consensus.cc:3058] T 5c3377ee8321447dacf77ca638c291e9 P b6c8f6ed18564f06b157bc0be92184bf [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:29.524713 15296 raft_consensus.cc:3058] T 5c3377ee8321447dacf77ca638c291e9 P f36679231d5d4b89bb3d043493457566 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:29.530880 15371 raft_consensus.cc:2466] T 5c3377ee8321447dacf77ca638c291e9 P b6c8f6ed18564f06b157bc0be92184bf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a9ee18d4024a57b8068cf872d335d2 in term 1.
I20250904 13:59:29.531064 15296 raft_consensus.cc:2466] T 5c3377ee8321447dacf77ca638c291e9 P f36679231d5d4b89bb3d043493457566 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40a9ee18d4024a57b8068cf872d335d2 in term 1.
I20250904 13:59:29.532044 15409 leader_election.cc:304] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40a9ee18d4024a57b8068cf872d335d2, b6c8f6ed18564f06b157bc0be92184bf; no voters:
I20250904 13:59:29.532820 15499 raft_consensus.cc:2802] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:29.533670 15499 raft_consensus.cc:695] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [term 1 LEADER]: Becoming Leader. State: Replica: 40a9ee18d4024a57b8068cf872d335d2, State: Running, Role: LEADER
I20250904 13:59:29.534904 15499 consensus_queue.cc:237] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c8f6ed18564f06b157bc0be92184bf" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41707 } } peers { permanent_uuid: "f36679231d5d4b89bb3d043493457566" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 44521 } } peers { permanent_uuid: "40a9ee18d4024a57b8068cf872d335d2" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 35351 } }
I20250904 13:59:29.543319 15193 catalog_manager.cc:5582] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40a9ee18d4024a57b8068cf872d335d2 (127.10.98.131). New cstate: current_term: 1 leader_uuid: "40a9ee18d4024a57b8068cf872d335d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6c8f6ed18564f06b157bc0be92184bf" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f36679231d5d4b89bb3d043493457566" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 44521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40a9ee18d4024a57b8068cf872d335d2" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 35351 } health_report { overall_health: HEALTHY } } }
I20250904 13:59:29.787856 15371 raft_consensus.cc:1273] T 5c3377ee8321447dacf77ca638c291e9 P b6c8f6ed18564f06b157bc0be92184bf [term 1 FOLLOWER]: Refusing update from remote peer 40a9ee18d4024a57b8068cf872d335d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 13:59:29.787856 15296 raft_consensus.cc:1273] T 5c3377ee8321447dacf77ca638c291e9 P f36679231d5d4b89bb3d043493457566 [term 1 FOLLOWER]: Refusing update from remote peer 40a9ee18d4024a57b8068cf872d335d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 13:59:29.789724 15502 consensus_queue.cc:1035] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6c8f6ed18564f06b157bc0be92184bf" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 41707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250904 13:59:29.790653 15499 consensus_queue.cc:1035] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f36679231d5d4b89bb3d043493457566" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 44521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:59:38.160950 10634 scanner-internal.cc:172] Scan of tablet 5c3377ee8321447dacf77ca638c291e9 at b6c8f6ed18564f06b157bc0be92184bf (127.10.98.130:41707) deadline exceeded after 1.000s
W20250904 13:59:38.161609 10634 scanner-internal.cc:458] Time spent opening tablet: real 1.254s user 0.008s sys 0.002s
I20250904 13:59:40.179021 10634 tablet_server.cc:178] TabletServer@127.10.98.129:0 shutting down...
I20250904 13:59:40.216914 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:59:40.217571 10634 tablet_replica.cc:331] T 5c3377ee8321447dacf77ca638c291e9 P f36679231d5d4b89bb3d043493457566: stopping tablet replica
I20250904 13:59:40.218178 10634 raft_consensus.cc:2241] T 5c3377ee8321447dacf77ca638c291e9 P f36679231d5d4b89bb3d043493457566 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:40.218883 10634 raft_consensus.cc:2270] T 5c3377ee8321447dacf77ca638c291e9 P f36679231d5d4b89bb3d043493457566 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:40.259822 10634 tablet_server.cc:195] TabletServer@127.10.98.129:0 shutdown complete.
I20250904 13:59:40.273185 10634 tablet_server.cc:178] TabletServer@127.10.98.130:0 shutting down...
I20250904 13:59:40.297027 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:59:40.297678 10634 tablet_replica.cc:331] T 5c3377ee8321447dacf77ca638c291e9 P b6c8f6ed18564f06b157bc0be92184bf: stopping tablet replica
I20250904 13:59:40.298300 10634 raft_consensus.cc:2241] T 5c3377ee8321447dacf77ca638c291e9 P b6c8f6ed18564f06b157bc0be92184bf [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:40.298897 10634 raft_consensus.cc:2270] T 5c3377ee8321447dacf77ca638c291e9 P b6c8f6ed18564f06b157bc0be92184bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:40.338034 10634 tablet_server.cc:195] TabletServer@127.10.98.130:0 shutdown complete.
I20250904 13:59:40.352846 10634 tablet_server.cc:178] TabletServer@127.10.98.131:0 shutting down...
I20250904 13:59:40.376909 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:59:40.377600 10634 tablet_replica.cc:331] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2: stopping tablet replica
I20250904 13:59:40.378298 10634 raft_consensus.cc:2241] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:40.379257 10634 raft_consensus.cc:2270] T 5c3377ee8321447dacf77ca638c291e9 P 40a9ee18d4024a57b8068cf872d335d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:40.417584 10634 tablet_server.cc:195] TabletServer@127.10.98.131:0 shutdown complete.
I20250904 13:59:40.432699 10634 master.cc:561] Master@127.10.98.190:39647 shutting down...
I20250904 13:59:40.461170 10634 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:40.462018 10634 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:40.462520 10634 tablet_replica.cc:331] T 00000000000000000000000000000000 P 2f31409816d64acb8621783fdbe35fd2: stopping tablet replica
I20250904 13:59:40.483177 10634 master.cc:583] Master@127.10.98.190:39647 shutdown complete.
[ OK ] NumReplicas/TServerQuiescingParamITest.TestScansRetry/1 (12105 ms)
[ RUN ] NumReplicas/TServerQuiescingParamITest.TestWriteWhileAllQuiescing/0
I20250904 13:59:40.521035 10634 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.98.190:36461
I20250904 13:59:40.522341 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:40.528209 15554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:40.528996 15555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:40.529812 15557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:40.530323 10634 server_base.cc:1047] running on GCE node
I20250904 13:59:40.531170 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:40.531387 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:40.531539 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994380531517 us; error 0 us; skew 500 ppm
I20250904 13:59:40.532051 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:40.534857 10634 webserver.cc:480] Webserver started at http://127.10.98.190:34269/ using document root <none> and password file <none>
I20250904 13:59:40.535387 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:40.535581 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:40.535856 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:40.537115 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_0.1756994241738102-10634-0/minicluster-data/master-0-root/instance:
uuid: "8ed0424792684fd5b380014276ab15b0"
format_stamp: "Formatted at 2025-09-04 13:59:40 on dist-test-slave-jx76"
I20250904 13:59:40.542317 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20250904 13:59:40.546152 15562 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:40.547091 10634 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20250904 13:59:40.547398 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_0.1756994241738102-10634-0/minicluster-data/master-0-root
uuid: "8ed0424792684fd5b380014276ab15b0"
format_stamp: "Formatted at 2025-09-04 13:59:40 on dist-test-slave-jx76"
I20250904 13:59:40.547691 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_0.1756994241738102-10634-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_0.1756994241738102-10634-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_0.1756994241738102-10634-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:40.571410 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:40.572714 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:40.618091 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.190:36461
I20250904 13:59:40.618255 15613 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.190:36461 every 8 connection(s)
I20250904 13:59:40.622416 15614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:40.634930 15614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0: Bootstrap starting.
I20250904 13:59:40.640107 15614 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:40.644970 15614 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0: No bootstrap required, opened a new log
I20250904 13:59:40.647210 15614 raft_consensus.cc:357] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed0424792684fd5b380014276ab15b0" member_type: VOTER }
I20250904 13:59:40.647707 15614 raft_consensus.cc:383] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:40.647897 15614 raft_consensus.cc:738] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ed0424792684fd5b380014276ab15b0, State: Initialized, Role: FOLLOWER
I20250904 13:59:40.648486 15614 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed0424792684fd5b380014276ab15b0" member_type: VOTER }
I20250904 13:59:40.648999 15614 raft_consensus.cc:397] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:59:40.649226 15614 raft_consensus.cc:491] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250904 13:59:40.649546 15614 raft_consensus.cc:3058] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:40.655011 15614 raft_consensus.cc:513] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed0424792684fd5b380014276ab15b0" member_type: VOTER }
I20250904 13:59:40.655567 15614 leader_election.cc:304] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ed0424792684fd5b380014276ab15b0; no voters:
I20250904 13:59:40.656819 15614 leader_election.cc:290] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:59:40.657272 15617 raft_consensus.cc:2802] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:40.658802 15617 raft_consensus.cc:695] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0 [term 1 LEADER]: Becoming Leader. State: Replica: 8ed0424792684fd5b380014276ab15b0, State: Running, Role: LEADER
I20250904 13:59:40.659652 15617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed0424792684fd5b380014276ab15b0" member_type: VOTER }
I20250904 13:59:40.660346 15614 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 13:59:40.665192 15619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ed0424792684fd5b380014276ab15b0. Latest consensus state: current_term: 1 leader_uuid: "8ed0424792684fd5b380014276ab15b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed0424792684fd5b380014276ab15b0" member_type: VOTER } }
I20250904 13:59:40.665343 15618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ed0424792684fd5b380014276ab15b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ed0424792684fd5b380014276ab15b0" member_type: VOTER } }
I20250904 13:59:40.666361 15619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0 [sys.catalog]: This master's current role is: LEADER
I20250904 13:59:40.666427 15618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0 [sys.catalog]: This master's current role is: LEADER
I20250904 13:59:40.673326 15625 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 13:59:40.679724 15625 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 13:59:40.681002 10634 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 13:59:40.689196 15625 catalog_manager.cc:1349] Generated new cluster ID: 194fb8d0a5a84c96b0bb5b4989148cf4
I20250904 13:59:40.689543 15625 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 13:59:40.708448 15625 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 13:59:40.709964 15625 catalog_manager.cc:1506] Loading token signing keys...
I20250904 13:59:40.729846 15625 catalog_manager.cc:5955] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0: Generated new TSK 0
I20250904 13:59:40.730643 15625 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 13:59:40.748467 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:40.755125 15635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:40.756769 15636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:40.759012 10634 server_base.cc:1047] running on GCE node
W20250904 13:59:40.758972 15638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:40.760244 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:40.760489 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:40.760654 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994380760633 us; error 0 us; skew 500 ppm
I20250904 13:59:40.761180 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:40.764325 10634 webserver.cc:480] Webserver started at http://127.10.98.129:36147/ using document root <none> and password file <none>
I20250904 13:59:40.764882 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:40.765081 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:40.765372 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:40.766685 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_0.1756994241738102-10634-0/minicluster-data/ts-0-root/instance:
uuid: "402096e0170c480db39665e0b300a783"
format_stamp: "Formatted at 2025-09-04 13:59:40 on dist-test-slave-jx76"
I20250904 13:59:40.771756 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20250904 13:59:40.775825 15643 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:40.776649 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20250904 13:59:40.776957 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_0.1756994241738102-10634-0/minicluster-data/ts-0-root
uuid: "402096e0170c480db39665e0b300a783"
format_stamp: "Formatted at 2025-09-04 13:59:40 on dist-test-slave-jx76"
I20250904 13:59:40.777278 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_0.1756994241738102-10634-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_0.1756994241738102-10634-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_0.1756994241738102-10634-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:40.793951 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:40.795367 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:40.797010 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:59:40.799681 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:59:40.799911 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:40.800184 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:59:40.800364 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:40.845114 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.129:36709
I20250904 13:59:40.845239 15705 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.129:36709 every 8 connection(s)
I20250904 13:59:40.866804 15706 heartbeater.cc:344] Connected to a master server at 127.10.98.190:36461
I20250904 13:59:40.867220 15706 heartbeater.cc:461] Registering TS with master...
I20250904 13:59:40.867993 15706 heartbeater.cc:507] Master 127.10.98.190:36461 requested a full tablet report, sending...
I20250904 13:59:40.870327 15579 ts_manager.cc:194] Registered new tserver with Master: 402096e0170c480db39665e0b300a783 (127.10.98.129:36709)
I20250904 13:59:40.871212 10634 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.022358343s
I20250904 13:59:40.872455 15579 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39070
I20250904 13:59:40.911839 10634 test_util.cc:276] Using random seed: 569529542
I20250904 13:59:40.936199 15579 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39104:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20250904 13:59:40.972110 15671 tablet_service.cc:1468] Processing CreateTablet for tablet 82f3f7d9a8d14828b09c8b690364b394 (DEFAULT_TABLE table=test-workload [id=8b4229d904f84effbafc7bb34c1a29d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:59:40.973582 15671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82f3f7d9a8d14828b09c8b690364b394. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:40.990041 15724 tablet_bootstrap.cc:492] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783: Bootstrap starting.
I20250904 13:59:40.996659 15724 tablet_bootstrap.cc:654] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:41.001364 15724 tablet_bootstrap.cc:492] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783: No bootstrap required, opened a new log
I20250904 13:59:41.001832 15724 ts_tablet_manager.cc:1397] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.007s
I20250904 13:59:41.004442 15724 raft_consensus.cc:357] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "402096e0170c480db39665e0b300a783" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36709 } }
I20250904 13:59:41.005059 15724 raft_consensus.cc:383] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:41.005352 15724 raft_consensus.cc:738] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 402096e0170c480db39665e0b300a783, State: Initialized, Role: FOLLOWER
I20250904 13:59:41.005918 15724 consensus_queue.cc:260] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "402096e0170c480db39665e0b300a783" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36709 } }
I20250904 13:59:41.006424 15724 raft_consensus.cc:397] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:59:41.006665 15724 raft_consensus.cc:491] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250904 13:59:41.006928 15724 raft_consensus.cc:3058] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:41.012977 15724 raft_consensus.cc:513] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "402096e0170c480db39665e0b300a783" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36709 } }
I20250904 13:59:41.013820 15724 leader_election.cc:304] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 402096e0170c480db39665e0b300a783; no voters:
I20250904 13:59:41.015483 15724 leader_election.cc:290] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:59:41.015928 15726 raft_consensus.cc:2802] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:41.017783 15706 heartbeater.cc:499] Master 127.10.98.190:36461 was elected leader, sending a full tablet report...
I20250904 13:59:41.018167 15724 ts_tablet_manager.cc:1428] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783: Time spent starting tablet: real 0.016s user 0.016s sys 0.001s
I20250904 13:59:41.018345 15726 raft_consensus.cc:695] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783 [term 1 LEADER]: Becoming Leader. State: Replica: 402096e0170c480db39665e0b300a783, State: Running, Role: LEADER
I20250904 13:59:41.019225 15726 consensus_queue.cc:237] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "402096e0170c480db39665e0b300a783" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36709 } }
I20250904 13:59:41.028999 15578 catalog_manager.cc:5582] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783 reported cstate change: term changed from 0 to 1, leader changed from <none> to 402096e0170c480db39665e0b300a783 (127.10.98.129). New cstate: current_term: 1 leader_uuid: "402096e0170c480db39665e0b300a783" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "402096e0170c480db39665e0b300a783" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 36709 } health_report { overall_health: HEALTHY } } }
I20250904 13:59:41.618348 10634 tablet_server.cc:178] TabletServer@127.10.98.129:0 shutting down...
I20250904 13:59:41.645411 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:59:41.646028 10634 tablet_replica.cc:331] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783: stopping tablet replica
I20250904 13:59:41.646593 10634 raft_consensus.cc:2241] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:41.647086 10634 raft_consensus.cc:2270] T 82f3f7d9a8d14828b09c8b690364b394 P 402096e0170c480db39665e0b300a783 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:41.669217 10634 tablet_server.cc:195] TabletServer@127.10.98.129:0 shutdown complete.
I20250904 13:59:41.680099 10634 master.cc:561] Master@127.10.98.190:36461 shutting down...
I20250904 13:59:41.699705 10634 raft_consensus.cc:2241] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:41.700245 10634 raft_consensus.cc:2270] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:41.700506 10634 tablet_replica.cc:331] T 00000000000000000000000000000000 P 8ed0424792684fd5b380014276ab15b0: stopping tablet replica
I20250904 13:59:41.719571 10634 master.cc:583] Master@127.10.98.190:36461 shutdown complete.
[ OK ] NumReplicas/TServerQuiescingParamITest.TestWriteWhileAllQuiescing/0 (1227 ms)
[ RUN ] NumReplicas/TServerQuiescingParamITest.TestWriteWhileAllQuiescing/1
I20250904 13:59:41.748239 10634 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.10.98.190:43517
I20250904 13:59:41.749475 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:41.754750 15738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:41.755005 15739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:41.756665 15741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:41.757578 10634 server_base.cc:1047] running on GCE node
I20250904 13:59:41.758404 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:41.758591 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:41.758752 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994381758731 us; error 0 us; skew 500 ppm
I20250904 13:59:41.759249 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:41.761646 10634 webserver.cc:480] Webserver started at http://127.10.98.190:40565/ using document root <none> and password file <none>
I20250904 13:59:41.762184 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:41.762375 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:41.762647 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:41.763806 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_1.1756994241738102-10634-0/minicluster-data/master-0-root/instance:
uuid: "f89dd12742b645abb158491f42637dc3"
format_stamp: "Formatted at 2025-09-04 13:59:41 on dist-test-slave-jx76"
I20250904 13:59:41.768608 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20250904 13:59:41.772472 15746 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:41.773240 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250904 13:59:41.773561 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_1.1756994241738102-10634-0/minicluster-data/master-0-root
uuid: "f89dd12742b645abb158491f42637dc3"
format_stamp: "Formatted at 2025-09-04 13:59:41 on dist-test-slave-jx76"
I20250904 13:59:41.773838 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_1.1756994241738102-10634-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_1.1756994241738102-10634-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_1.1756994241738102-10634-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:41.798753 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:41.799824 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:41.837568 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.190:43517
I20250904 13:59:41.837663 15797 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.190:43517 every 8 connection(s)
I20250904 13:59:41.841748 15798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:41.854086 15798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3: Bootstrap starting.
I20250904 13:59:41.858814 15798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:41.863219 15798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3: No bootstrap required, opened a new log
I20250904 13:59:41.865228 15798 raft_consensus.cc:357] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89dd12742b645abb158491f42637dc3" member_type: VOTER }
I20250904 13:59:41.865629 15798 raft_consensus.cc:383] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:41.865823 15798 raft_consensus.cc:738] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f89dd12742b645abb158491f42637dc3, State: Initialized, Role: FOLLOWER
I20250904 13:59:41.866367 15798 consensus_queue.cc:260] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89dd12742b645abb158491f42637dc3" member_type: VOTER }
I20250904 13:59:41.866811 15798 raft_consensus.cc:397] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250904 13:59:41.866999 15798 raft_consensus.cc:491] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250904 13:59:41.867214 15798 raft_consensus.cc:3058] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:41.872596 15798 raft_consensus.cc:513] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89dd12742b645abb158491f42637dc3" member_type: VOTER }
I20250904 13:59:41.873137 15798 leader_election.cc:304] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f89dd12742b645abb158491f42637dc3; no voters:
I20250904 13:59:41.874433 15798 leader_election.cc:290] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250904 13:59:41.874787 15801 raft_consensus.cc:2802] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:41.876296 15801 raft_consensus.cc:695] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3 [term 1 LEADER]: Becoming Leader. State: Replica: f89dd12742b645abb158491f42637dc3, State: Running, Role: LEADER
I20250904 13:59:41.877020 15801 consensus_queue.cc:237] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89dd12742b645abb158491f42637dc3" member_type: VOTER }
I20250904 13:59:41.877823 15798 sys_catalog.cc:564] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3 [sys.catalog]: configured and running, proceeding with master startup.
I20250904 13:59:41.880136 15802 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f89dd12742b645abb158491f42637dc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89dd12742b645abb158491f42637dc3" member_type: VOTER } }
I20250904 13:59:41.880311 15803 sys_catalog.cc:455] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f89dd12742b645abb158491f42637dc3. Latest consensus state: current_term: 1 leader_uuid: "f89dd12742b645abb158491f42637dc3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f89dd12742b645abb158491f42637dc3" member_type: VOTER } }
I20250904 13:59:41.880975 15803 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3 [sys.catalog]: This master's current role is: LEADER
I20250904 13:59:41.880954 15802 sys_catalog.cc:458] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3 [sys.catalog]: This master's current role is: LEADER
I20250904 13:59:41.885211 15807 catalog_manager.cc:1477] Loading table and tablet metadata into memory...
I20250904 13:59:41.889966 15807 catalog_manager.cc:1486] Initializing Kudu cluster ID...
I20250904 13:59:41.895045 10634 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20250904 13:59:41.899457 15807 catalog_manager.cc:1349] Generated new cluster ID: fddf0ab832c44dfc9615a214f5ca914f
I20250904 13:59:41.899797 15807 catalog_manager.cc:1497] Initializing Kudu internal certificate authority...
I20250904 13:59:41.931598 15807 catalog_manager.cc:1372] Generated new certificate authority record
I20250904 13:59:41.933068 15807 catalog_manager.cc:1506] Loading token signing keys...
I20250904 13:59:41.946556 15807 catalog_manager.cc:5955] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3: Generated new TSK 0
I20250904 13:59:41.947271 15807 catalog_manager.cc:1516] Initializing in-progress tserver states...
I20250904 13:59:41.963114 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:41.969195 15819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:41.970395 15820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:41.971998 15822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:41.972064 10634 server_base.cc:1047] running on GCE node
I20250904 13:59:41.973089 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:41.973308 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:41.973460 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994381973441 us; error 0 us; skew 500 ppm
I20250904 13:59:41.973934 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:41.976402 10634 webserver.cc:480] Webserver started at http://127.10.98.129:35933/ using document root <none> and password file <none>
I20250904 13:59:41.976895 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:41.977082 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:41.977350 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:41.978586 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_1.1756994241738102-10634-0/minicluster-data/ts-0-root/instance:
uuid: "2f0ae914144d4dec9980b694c28cfb82"
format_stamp: "Formatted at 2025-09-04 13:59:41 on dist-test-slave-jx76"
I20250904 13:59:41.983467 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20250904 13:59:41.987187 15827 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:41.988116 10634 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20250904 13:59:41.988512 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_1.1756994241738102-10634-0/minicluster-data/ts-0-root
uuid: "2f0ae914144d4dec9980b694c28cfb82"
format_stamp: "Formatted at 2025-09-04 13:59:41 on dist-test-slave-jx76"
I20250904 13:59:41.988839 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_1.1756994241738102-10634-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_1.1756994241738102-10634-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_1.1756994241738102-10634-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:42.015667 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:42.016706 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:42.018218 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:59:42.020637 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:59:42.020841 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:42.021111 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:59:42.021260 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:42.062187 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.129:45137
I20250904 13:59:42.062346 15889 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.129:45137 every 8 connection(s)
I20250904 13:59:42.067490 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:42.074666 15894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:42.075814 15895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:42.079034 15897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:42.079634 10634 server_base.cc:1047] running on GCE node
I20250904 13:59:42.080667 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:42.080945 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:42.081138 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994382081118 us; error 0 us; skew 500 ppm
I20250904 13:59:42.081250 15890 heartbeater.cc:344] Connected to a master server at 127.10.98.190:43517
I20250904 13:59:42.081636 15890 heartbeater.cc:461] Registering TS with master...
I20250904 13:59:42.081696 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:42.082492 15890 heartbeater.cc:507] Master 127.10.98.190:43517 requested a full tablet report, sending...
I20250904 13:59:42.084640 10634 webserver.cc:480] Webserver started at http://127.10.98.130:33591/ using document root <none> and password file <none>
I20250904 13:59:42.085106 15763 ts_manager.cc:194] Registered new tserver with Master: 2f0ae914144d4dec9980b694c28cfb82 (127.10.98.129:45137)
I20250904 13:59:42.085369 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:42.085687 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:42.086025 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:42.087610 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_1.1756994241738102-10634-0/minicluster-data/ts-1-root/instance:
uuid: "9382eefd72ee40c096ec43d5191ac437"
format_stamp: "Formatted at 2025-09-04 13:59:42 on dist-test-slave-jx76"
I20250904 13:59:42.087723 15763 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46004
I20250904 13:59:42.093092 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20250904 13:59:42.096858 15902 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:42.097820 10634 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20250904 13:59:42.098156 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_1.1756994241738102-10634-0/minicluster-data/ts-1-root
uuid: "9382eefd72ee40c096ec43d5191ac437"
format_stamp: "Formatted at 2025-09-04 13:59:42 on dist-test-slave-jx76"
I20250904 13:59:42.098464 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_1.1756994241738102-10634-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_1.1756994241738102-10634-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_1.1756994241738102-10634-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:42.113343 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:42.114569 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:42.116132 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:59:42.118786 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:59:42.118993 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:42.119266 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:59:42.119408 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:42.161525 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.130:44407
I20250904 13:59:42.161628 15964 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.130:44407 every 8 connection(s)
I20250904 13:59:42.166639 10634 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250904 13:59:42.173164 15968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:42.176618 10634 server_base.cc:1047] running on GCE node
W20250904 13:59:42.176522 15969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250904 13:59:42.178283 15971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250904 13:59:42.179528 10634 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20250904 13:59:42.179816 10634 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20250904 13:59:42.179994 10634 hybrid_clock.cc:648] HybridClock initialized: now 1756994382179978 us; error 0 us; skew 500 ppm
I20250904 13:59:42.180199 15965 heartbeater.cc:344] Connected to a master server at 127.10.98.190:43517
I20250904 13:59:42.180680 15965 heartbeater.cc:461] Registering TS with master...
I20250904 13:59:42.180774 10634 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250904 13:59:42.181623 15965 heartbeater.cc:507] Master 127.10.98.190:43517 requested a full tablet report, sending...
I20250904 13:59:42.183755 10634 webserver.cc:480] Webserver started at http://127.10.98.131:40531/ using document root <none> and password file <none>
I20250904 13:59:42.184223 15763 ts_manager.cc:194] Registered new tserver with Master: 9382eefd72ee40c096ec43d5191ac437 (127.10.98.130:44407)
I20250904 13:59:42.184471 10634 fs_manager.cc:362] Metadata directory not provided
I20250904 13:59:42.184751 10634 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250904 13:59:42.185075 10634 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20250904 13:59:42.186281 15763 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46012
I20250904 13:59:42.186874 10634 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_1.1756994241738102-10634-0/minicluster-data/ts-2-root/instance:
uuid: "4e05ade39d5b46eea85d8cb8df6b1af5"
format_stamp: "Formatted at 2025-09-04 13:59:42 on dist-test-slave-jx76"
I20250904 13:59:42.192255 10634 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20250904 13:59:42.195904 15976 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:42.196694 10634 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20250904 13:59:42.197002 10634 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_1.1756994241738102-10634-0/minicluster-data/ts-2-root
uuid: "4e05ade39d5b46eea85d8cb8df6b1af5"
format_stamp: "Formatted at 2025-09-04 13:59:42 on dist-test-slave-jx76"
I20250904 13:59:42.197307 10634 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_1.1756994241738102-10634-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_1.1756994241738102-10634-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskIqzzAZ/test-tmp/tablet_server_quiescing-itest.0.NumReplicas_TServerQuiescingParamITest.TestWriteWhileAllQuiescing_1.1756994241738102-10634-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250904 13:59:42.210076 10634 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250904 13:59:42.211402 10634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250904 13:59:42.212981 10634 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250904 13:59:42.215507 10634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250904 13:59:42.215721 10634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:42.215953 10634 ts_tablet_manager.cc:610] Registered 0 tablets
I20250904 13:59:42.216111 10634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250904 13:59:42.258422 10634 rpc_server.cc:307] RPC server started. Bound to: 127.10.98.131:36505
I20250904 13:59:42.258535 16038 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.10.98.131:36505 every 8 connection(s)
I20250904 13:59:42.273283 16039 heartbeater.cc:344] Connected to a master server at 127.10.98.190:43517
I20250904 13:59:42.273658 16039 heartbeater.cc:461] Registering TS with master...
I20250904 13:59:42.274458 16039 heartbeater.cc:507] Master 127.10.98.190:43517 requested a full tablet report, sending...
I20250904 13:59:42.276516 15763 ts_manager.cc:194] Registered new tserver with Master: 4e05ade39d5b46eea85d8cb8df6b1af5 (127.10.98.131:36505)
I20250904 13:59:42.276793 10634 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014839261s
I20250904 13:59:42.278007 15763 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46024
I20250904 13:59:42.311983 10634 test_util.cc:276] Using random seed: 570929684
I20250904 13:59:42.336014 15763 catalog_manager.cc:2232] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46038:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250904 13:59:42.337874 15763 catalog_manager.cc:6944] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250904 13:59:42.374298 15855 tablet_service.cc:1468] Processing CreateTablet for tablet 4202dc26544645b8bc4a4fec837100d3 (DEFAULT_TABLE table=test-workload [id=11eb1f78346f4c35ab43ce3ac25f413c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:59:42.375823 15855 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4202dc26544645b8bc4a4fec837100d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:42.379792 15930 tablet_service.cc:1468] Processing CreateTablet for tablet 4202dc26544645b8bc4a4fec837100d3 (DEFAULT_TABLE table=test-workload [id=11eb1f78346f4c35ab43ce3ac25f413c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:59:42.380265 16004 tablet_service.cc:1468] Processing CreateTablet for tablet 4202dc26544645b8bc4a4fec837100d3 (DEFAULT_TABLE table=test-workload [id=11eb1f78346f4c35ab43ce3ac25f413c]), partition=RANGE (key) PARTITION UNBOUNDED
I20250904 13:59:42.381246 15930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4202dc26544645b8bc4a4fec837100d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:42.381671 16004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4202dc26544645b8bc4a4fec837100d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20250904 13:59:42.397799 16060 tablet_bootstrap.cc:492] T 4202dc26544645b8bc4a4fec837100d3 P 9382eefd72ee40c096ec43d5191ac437: Bootstrap starting.
I20250904 13:59:42.401392 16061 tablet_bootstrap.cc:492] T 4202dc26544645b8bc4a4fec837100d3 P 2f0ae914144d4dec9980b694c28cfb82: Bootstrap starting.
I20250904 13:59:42.404685 16062 tablet_bootstrap.cc:492] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5: Bootstrap starting.
I20250904 13:59:42.405886 16060 tablet_bootstrap.cc:654] T 4202dc26544645b8bc4a4fec837100d3 P 9382eefd72ee40c096ec43d5191ac437: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:42.408814 16061 tablet_bootstrap.cc:654] T 4202dc26544645b8bc4a4fec837100d3 P 2f0ae914144d4dec9980b694c28cfb82: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:42.411464 16062 tablet_bootstrap.cc:654] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5: Neither blocks nor log segments found. Creating new log.
I20250904 13:59:42.414770 16060 tablet_bootstrap.cc:492] T 4202dc26544645b8bc4a4fec837100d3 P 9382eefd72ee40c096ec43d5191ac437: No bootstrap required, opened a new log
I20250904 13:59:42.415287 16060 ts_tablet_manager.cc:1397] T 4202dc26544645b8bc4a4fec837100d3 P 9382eefd72ee40c096ec43d5191ac437: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.005s
I20250904 13:59:42.416803 16061 tablet_bootstrap.cc:492] T 4202dc26544645b8bc4a4fec837100d3 P 2f0ae914144d4dec9980b694c28cfb82: No bootstrap required, opened a new log
I20250904 13:59:42.417431 16061 ts_tablet_manager.cc:1397] T 4202dc26544645b8bc4a4fec837100d3 P 2f0ae914144d4dec9980b694c28cfb82: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.000s
I20250904 13:59:42.418224 16060 raft_consensus.cc:357] T 4202dc26544645b8bc4a4fec837100d3 P 9382eefd72ee40c096ec43d5191ac437 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0ae914144d4dec9980b694c28cfb82" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 45137 } } peers { permanent_uuid: "9382eefd72ee40c096ec43d5191ac437" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 44407 } } peers { permanent_uuid: "4e05ade39d5b46eea85d8cb8df6b1af5" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 36505 } }
I20250904 13:59:42.418604 16062 tablet_bootstrap.cc:492] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5: No bootstrap required, opened a new log
I20250904 13:59:42.418880 16060 raft_consensus.cc:383] T 4202dc26544645b8bc4a4fec837100d3 P 9382eefd72ee40c096ec43d5191ac437 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:42.419196 16060 raft_consensus.cc:738] T 4202dc26544645b8bc4a4fec837100d3 P 9382eefd72ee40c096ec43d5191ac437 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9382eefd72ee40c096ec43d5191ac437, State: Initialized, Role: FOLLOWER
I20250904 13:59:42.419232 16062 ts_tablet_manager.cc:1397] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5: Time spent bootstrapping tablet: real 0.015s user 0.014s sys 0.000s
I20250904 13:59:42.419925 16060 consensus_queue.cc:260] T 4202dc26544645b8bc4a4fec837100d3 P 9382eefd72ee40c096ec43d5191ac437 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0ae914144d4dec9980b694c28cfb82" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 45137 } } peers { permanent_uuid: "9382eefd72ee40c096ec43d5191ac437" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 44407 } } peers { permanent_uuid: "4e05ade39d5b46eea85d8cb8df6b1af5" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 36505 } }
I20250904 13:59:42.420754 16061 raft_consensus.cc:357] T 4202dc26544645b8bc4a4fec837100d3 P 2f0ae914144d4dec9980b694c28cfb82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0ae914144d4dec9980b694c28cfb82" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 45137 } } peers { permanent_uuid: "9382eefd72ee40c096ec43d5191ac437" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 44407 } } peers { permanent_uuid: "4e05ade39d5b46eea85d8cb8df6b1af5" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 36505 } }
I20250904 13:59:42.421670 16061 raft_consensus.cc:383] T 4202dc26544645b8bc4a4fec837100d3 P 2f0ae914144d4dec9980b694c28cfb82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:42.422032 16061 raft_consensus.cc:738] T 4202dc26544645b8bc4a4fec837100d3 P 2f0ae914144d4dec9980b694c28cfb82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f0ae914144d4dec9980b694c28cfb82, State: Initialized, Role: FOLLOWER
I20250904 13:59:42.422574 16062 raft_consensus.cc:357] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0ae914144d4dec9980b694c28cfb82" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 45137 } } peers { permanent_uuid: "9382eefd72ee40c096ec43d5191ac437" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 44407 } } peers { permanent_uuid: "4e05ade39d5b46eea85d8cb8df6b1af5" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 36505 } }
I20250904 13:59:42.422978 16061 consensus_queue.cc:260] T 4202dc26544645b8bc4a4fec837100d3 P 2f0ae914144d4dec9980b694c28cfb82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0ae914144d4dec9980b694c28cfb82" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 45137 } } peers { permanent_uuid: "9382eefd72ee40c096ec43d5191ac437" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 44407 } } peers { permanent_uuid: "4e05ade39d5b46eea85d8cb8df6b1af5" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 36505 } }
I20250904 13:59:42.423599 16062 raft_consensus.cc:383] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250904 13:59:42.424039 16062 raft_consensus.cc:738] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e05ade39d5b46eea85d8cb8df6b1af5, State: Initialized, Role: FOLLOWER
I20250904 13:59:42.425292 16062 consensus_queue.cc:260] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0ae914144d4dec9980b694c28cfb82" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 45137 } } peers { permanent_uuid: "9382eefd72ee40c096ec43d5191ac437" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 44407 } } peers { permanent_uuid: "4e05ade39d5b46eea85d8cb8df6b1af5" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 36505 } }
I20250904 13:59:42.426820 15965 heartbeater.cc:499] Master 127.10.98.190:43517 was elected leader, sending a full tablet report...
I20250904 13:59:42.427168 16060 ts_tablet_manager.cc:1428] T 4202dc26544645b8bc4a4fec837100d3 P 9382eefd72ee40c096ec43d5191ac437: Time spent starting tablet: real 0.011s user 0.000s sys 0.010s
I20250904 13:59:42.429836 15890 heartbeater.cc:499] Master 127.10.98.190:43517 was elected leader, sending a full tablet report...
I20250904 13:59:42.431150 16061 ts_tablet_manager.cc:1428] T 4202dc26544645b8bc4a4fec837100d3 P 2f0ae914144d4dec9980b694c28cfb82: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20250904 13:59:42.435295 16039 heartbeater.cc:499] Master 127.10.98.190:43517 was elected leader, sending a full tablet report...
I20250904 13:59:42.436092 16062 ts_tablet_manager.cc:1428] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
W20250904 13:59:42.512812 16040 tablet.cc:2378] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250904 13:59:42.530731 16068 raft_consensus.cc:491] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250904 13:59:42.531308 16068 raft_consensus.cc:513] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0ae914144d4dec9980b694c28cfb82" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 45137 } } peers { permanent_uuid: "9382eefd72ee40c096ec43d5191ac437" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 44407 } } peers { permanent_uuid: "4e05ade39d5b46eea85d8cb8df6b1af5" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 36505 } }
I20250904 13:59:42.536023 16068 leader_election.cc:290] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f0ae914144d4dec9980b694c28cfb82 (127.10.98.129:45137), 9382eefd72ee40c096ec43d5191ac437 (127.10.98.130:44407)
I20250904 13:59:42.544303 15865 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4202dc26544645b8bc4a4fec837100d3" candidate_uuid: "4e05ade39d5b46eea85d8cb8df6b1af5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f0ae914144d4dec9980b694c28cfb82" is_pre_election: true
I20250904 13:59:42.545312 15865 raft_consensus.cc:2466] T 4202dc26544645b8bc4a4fec837100d3 P 2f0ae914144d4dec9980b694c28cfb82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e05ade39d5b46eea85d8cb8df6b1af5 in term 0.
I20250904 13:59:42.545809 15940 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4202dc26544645b8bc4a4fec837100d3" candidate_uuid: "4e05ade39d5b46eea85d8cb8df6b1af5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9382eefd72ee40c096ec43d5191ac437" is_pre_election: true
I20250904 13:59:42.546550 15979 leader_election.cc:304] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f0ae914144d4dec9980b694c28cfb82, 4e05ade39d5b46eea85d8cb8df6b1af5; no voters:
I20250904 13:59:42.546571 15940 raft_consensus.cc:2466] T 4202dc26544645b8bc4a4fec837100d3 P 9382eefd72ee40c096ec43d5191ac437 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e05ade39d5b46eea85d8cb8df6b1af5 in term 0.
I20250904 13:59:42.547358 16068 raft_consensus.cc:2802] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250904 13:59:42.547745 16068 raft_consensus.cc:491] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250904 13:59:42.548017 16068 raft_consensus.cc:3058] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:42.553790 16068 raft_consensus.cc:513] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0ae914144d4dec9980b694c28cfb82" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 45137 } } peers { permanent_uuid: "9382eefd72ee40c096ec43d5191ac437" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 44407 } } peers { permanent_uuid: "4e05ade39d5b46eea85d8cb8df6b1af5" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 36505 } }
I20250904 13:59:42.555650 16068 leader_election.cc:290] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [CANDIDATE]: Term 1 election: Requested vote from peers 2f0ae914144d4dec9980b694c28cfb82 (127.10.98.129:45137), 9382eefd72ee40c096ec43d5191ac437 (127.10.98.130:44407)
I20250904 13:59:42.556689 15865 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4202dc26544645b8bc4a4fec837100d3" candidate_uuid: "4e05ade39d5b46eea85d8cb8df6b1af5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f0ae914144d4dec9980b694c28cfb82"
I20250904 13:59:42.556844 15940 tablet_service.cc:1813] Received RequestConsensusVote() RPC: tablet_id: "4202dc26544645b8bc4a4fec837100d3" candidate_uuid: "4e05ade39d5b46eea85d8cb8df6b1af5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9382eefd72ee40c096ec43d5191ac437"
I20250904 13:59:42.557221 15865 raft_consensus.cc:3058] T 4202dc26544645b8bc4a4fec837100d3 P 2f0ae914144d4dec9980b694c28cfb82 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:42.557369 15940 raft_consensus.cc:3058] T 4202dc26544645b8bc4a4fec837100d3 P 9382eefd72ee40c096ec43d5191ac437 [term 0 FOLLOWER]: Advancing to term 1
I20250904 13:59:42.563540 15940 raft_consensus.cc:2466] T 4202dc26544645b8bc4a4fec837100d3 P 9382eefd72ee40c096ec43d5191ac437 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e05ade39d5b46eea85d8cb8df6b1af5 in term 1.
I20250904 13:59:42.563540 15865 raft_consensus.cc:2466] T 4202dc26544645b8bc4a4fec837100d3 P 2f0ae914144d4dec9980b694c28cfb82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e05ade39d5b46eea85d8cb8df6b1af5 in term 1.
I20250904 13:59:42.564936 15978 leader_election.cc:304] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e05ade39d5b46eea85d8cb8df6b1af5, 9382eefd72ee40c096ec43d5191ac437; no voters:
I20250904 13:59:42.565734 16068 raft_consensus.cc:2802] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [term 1 FOLLOWER]: Leader election won for term 1
I20250904 13:59:42.566838 16068 raft_consensus.cc:695] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [term 1 LEADER]: Becoming Leader. State: Replica: 4e05ade39d5b46eea85d8cb8df6b1af5, State: Running, Role: LEADER
I20250904 13:59:42.567751 16068 consensus_queue.cc:237] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0ae914144d4dec9980b694c28cfb82" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 45137 } } peers { permanent_uuid: "9382eefd72ee40c096ec43d5191ac437" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 44407 } } peers { permanent_uuid: "4e05ade39d5b46eea85d8cb8df6b1af5" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 36505 } }
I20250904 13:59:42.575560 15761 catalog_manager.cc:5582] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e05ade39d5b46eea85d8cb8df6b1af5 (127.10.98.131). New cstate: current_term: 1 leader_uuid: "4e05ade39d5b46eea85d8cb8df6b1af5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f0ae914144d4dec9980b694c28cfb82" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 45137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9382eefd72ee40c096ec43d5191ac437" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 44407 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e05ade39d5b46eea85d8cb8df6b1af5" member_type: VOTER last_known_addr { host: "127.10.98.131" port: 36505 } health_report { overall_health: HEALTHY } } }
I20250904 13:59:42.788515 15865 raft_consensus.cc:1273] T 4202dc26544645b8bc4a4fec837100d3 P 2f0ae914144d4dec9980b694c28cfb82 [term 1 FOLLOWER]: Refusing update from remote peer 4e05ade39d5b46eea85d8cb8df6b1af5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 13:59:42.788519 15940 raft_consensus.cc:1273] T 4202dc26544645b8bc4a4fec837100d3 P 9382eefd72ee40c096ec43d5191ac437 [term 1 FOLLOWER]: Refusing update from remote peer 4e05ade39d5b46eea85d8cb8df6b1af5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250904 13:59:42.790499 16070 consensus_queue.cc:1035] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f0ae914144d4dec9980b694c28cfb82" member_type: VOTER last_known_addr { host: "127.10.98.129" port: 45137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20250904 13:59:42.791328 16068 consensus_queue.cc:1035] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9382eefd72ee40c096ec43d5191ac437" member_type: VOTER last_known_addr { host: "127.10.98.130" port: 44407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250904 13:59:42.808689 16076 mvcc.cc:204] Tried to move back new op lower bound from 7196648991886532608 to 7196648991004553216. Current Snapshot: MvccSnapshot[applied={T|T < 7196648991886532608}]
I20250904 13:59:43.829541 10634 tablet_server.cc:178] TabletServer@127.10.98.129:0 shutting down...
I20250904 13:59:43.867790 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:59:43.868583 10634 tablet_replica.cc:331] T 4202dc26544645b8bc4a4fec837100d3 P 2f0ae914144d4dec9980b694c28cfb82: stopping tablet replica
I20250904 13:59:43.869624 10634 raft_consensus.cc:2241] T 4202dc26544645b8bc4a4fec837100d3 P 2f0ae914144d4dec9980b694c28cfb82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:43.870381 10634 raft_consensus.cc:2270] T 4202dc26544645b8bc4a4fec837100d3 P 2f0ae914144d4dec9980b694c28cfb82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:43.897574 10634 tablet_server.cc:195] TabletServer@127.10.98.129:0 shutdown complete.
I20250904 13:59:43.913271 10634 tablet_server.cc:178] TabletServer@127.10.98.130:0 shutting down...
I20250904 13:59:43.944619 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:59:43.945358 10634 tablet_replica.cc:331] T 4202dc26544645b8bc4a4fec837100d3 P 9382eefd72ee40c096ec43d5191ac437: stopping tablet replica
I20250904 13:59:43.946014 10634 raft_consensus.cc:2241] T 4202dc26544645b8bc4a4fec837100d3 P 9382eefd72ee40c096ec43d5191ac437 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250904 13:59:43.946724 10634 raft_consensus.cc:2270] T 4202dc26544645b8bc4a4fec837100d3 P 9382eefd72ee40c096ec43d5191ac437 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:43.960692 10634 tablet_server.cc:195] TabletServer@127.10.98.130:0 shutdown complete.
I20250904 13:59:43.973891 10634 tablet_server.cc:178] TabletServer@127.10.98.131:0 shutting down...
I20250904 13:59:43.999630 10634 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20250904 13:59:44.000356 10634 tablet_replica.cc:331] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5: stopping tablet replica
I20250904 13:59:44.000967 10634 raft_consensus.cc:2241] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:44.002071 10634 raft_consensus.cc:2270] T 4202dc26544645b8bc4a4fec837100d3 P 4e05ade39d5b46eea85d8cb8df6b1af5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:44.026820 10634 tablet_server.cc:195] TabletServer@127.10.98.131:0 shutdown complete.
I20250904 13:59:44.040449 10634 master.cc:561] Master@127.10.98.190:43517 shutting down...
I20250904 13:59:44.066489 10634 raft_consensus.cc:2241] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3 [term 1 LEADER]: Raft consensus shutting down.
I20250904 13:59:44.067232 10634 raft_consensus.cc:2270] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250904 13:59:44.067595 10634 tablet_replica.cc:331] T 00000000000000000000000000000000 P f89dd12742b645abb158491f42637dc3: stopping tablet replica
I20250904 13:59:44.088248 10634 master.cc:583] Master@127.10.98.190:43517 shutdown complete.
[ OK ] NumReplicas/TServerQuiescingParamITest.TestWriteWhileAllQuiescing/1 (2377 ms)
[ RUN ] NumReplicas/TServerQuiescingParamITest.TestAbruptStepdownWhileAllQuiescing/0
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/tablet_server_quiescing-itest.cc:730: Skipped
test is skipped; set KUDU_ALLOW_SLOW_TESTS=1 to run
[ SKIPPED ] NumReplicas/TServerQuiescingParamITest.TestAbruptStepdownWhileAllQuiescing/0 (8 ms)
[ RUN ] NumReplicas/TServerQuiescingParamITest.TestAbruptStepdownWhileAllQuiescing/1
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/tablet_server_quiescing-itest.cc:730: Skipped
test is skipped; set KUDU_ALLOW_SLOW_TESTS=1 to run
[ SKIPPED ] NumReplicas/TServerQuiescingParamITest.TestAbruptStepdownWhileAllQuiescing/1 (8 ms)
[----------] 10 tests from NumReplicas/TServerQuiescingParamITest (37701 ms total)
[----------] Global test environment tear-down
[==========] 18 tests from 2 test suites ran. (142231 ms total)
[ PASSED ] 16 tests.
[ SKIPPED ] 2 tests, listed below:
[ SKIPPED ] NumReplicas/TServerQuiescingParamITest.TestAbruptStepdownWhileAllQuiescing/0
[ SKIPPED ] NumReplicas/TServerQuiescingParamITest.TestAbruptStepdownWhileAllQuiescing/1
I20250904 13:59:44.151949 10634 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/raft_consensus.cc:668: suppressed but not reported on 39 messages since previous log ~33 seconds ago
I20250904 13:59:44.152238 10634 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/raft_consensus.cc:399: suppressed but not reported on 9 messages since previous log ~33 seconds ago
I20250904 13:59:44.152491 10634 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/meta_cache.cc:302: suppressed but not reported on 5 messages since previous log ~126 seconds ago
I20250904 13:59:44.152706 10634 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 3 messages since previous log ~26 seconds ago
I20250904 13:59:44.153033 10634 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet.cc:2378: suppressed but not reported on 2 messages since previous log ~1 seconds ago
ThreadSanitizer: reported 2 warnings